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Force-Proportional  Reinforcement  With 
An  Application  of  Spectral  Analysis 

Stephen  C.  Fowler 

A  force-proportional  reinforcement  schedule  specifies  the 
relation  between  the  peak  force  of  a  bar  press  and  the  amount 

t 

of  reinforcement  obtained  by  such  a  response.  Three  experiments 
were  undertaken  and  detailed  statistical  analyses  (linear  re¬ 
gression  analysis  and  spectral  analysis)  were  conducted  in  order 
to  describe  the  effects  of  proportional  reinforcement  on  force 
of  bar  pressing.  Hungry  rats  were  trained  to  exert  pressure  on 
a  silent,  isometric  manipulandum  and  were  rewarded  with  sucrose 
solutions  according  to  a  variety  of  parametric  arrangements. 
Several  hundred  responses  were  measured  for  each  of  the  66  rats 
used  in  the  experiments. 

Specifically,  the  research  addressed  itself  to  these  three 
questions: 

1.  Do  hungry  rats  “take  advantage"  of  the  force  propor- 

\ 

tional  reinforcement  contingency  by  emitting  higher  forces  to 
obtain  greater  amounts  of  reinforcement? 

2.  Is  the  ratio  of  exertion  to  payoff  (specified  gain) 
important  in  determining  the  degree  to  which  proportional  rein¬ 
forcement  influences  response  force? 

3.  Do  the  sequences  of  response  forces,  emitted  within 
a  session  of  responding,  display  any  noteworthy  trends  or  serial 
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dependencies  which  might  be  interpreted  as  indicating  the 
presence  of  response  -  and  reinforcement  -  induced  feedback? 

In  the  first  experiment/  access  time  to  a  sucrose  solution 
was  made  continuously  proportional  to  the  force  of  response. 
Measurements  of  lick  rate  and  volume  consumed  demonstrated  that 
continuous  proportional  reinforcement  was  operative.  Despite 
successful  implementation  of  this  schedule,  the  rats  displayed 
no  proclivity  to  press  harder  to  obtain  more  of  the  same  con¬ 
centration  of  sucrose.  This  latter  result  was  consistent  with 
previous  research  which  used  food  pellets  as  reinforcement  in 
discontinuous,  step-wise  proportional  schedules  (Notterman  & 
Mintz,  1965;  Filion,  Fowler,  &  Notterman,  1969;  Filion,  Fowler, 
&Notterman,  1970). 

In  a  second  experiment  force  proportional  reinforcement  was 
specified  by  concentration  of  sucrose  so  that  more  forceful 
bar  presses  produced  sweeter  solutions.  As  a  result  of  exposure 
to  this  type  of  contingency,  most  rats  did  learn  to  respond  with 
greater  forces. 

A  third  experiment  extended  the  force-proportional  concen¬ 
tration  schedule  to  an  examination  of  the  effects  of  gain  on 
peak  force.  Within  the  limits  of  the  parametric  conditions  used 
gain  did  not  appear  to  be  a  potent  independent  variable.  Indi¬ 
vidual  differences  and  "least  effort"  tendencies  may  be  more 
important  than  gain  in  determining  peak  force  of  response. 

Data  gathered  in  the  second  and  third  experiments  were 
examined  by  means  of  linear  regression  analysis  and  spectral 
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analysis.  The  slopes  of  the  regression  lines  of  peak  force  on 

ordinal  position  did  not  depart  significantly  from  zero.  (From 

50  to  100  responses  were  used  in  computing  the  within-session 

slopes  for  each  of  the  49  rats.  Spectral  analyses  were  also 

performed  on  these  data.)  Thus  linear  regression  analysis  afforded 

no  evidence  for  "slow"  changes  in  peak  force  that  might  have 

been  expected  on  the  basis  of  learning  or  drive  reduction. 

Following  the  lead  of  Abelson  (1953)  and  using  the  procedures 

outlined  by  Jenkins  &  Watts  (1968) ,  spectral  analysis  was  applied 

to  the  session  peak  force  records  in  order  to  detect  possible 

serial  dependencies  that  might  arise  from  changes  in  concentration 

of  sucrose  encountered  by  a  rat  responding  on  a  proportional 
/ 

reinforcement  schedule.  The  sequences  of  responses  emitted  during 
a  session  showed  only  slight  departure  from  randomness,  in  the 
sense  that  the  estimated  spectral  density  functions  were  gener¬ 
ally  flat.  Such  non-random  effects  that  did  emerge  [ parti¬ 
cular  individuals  took  the  form  of  a  high  frequency  component 
(.33  to  .5  cycles  per  response),  indicating  a  tendency  for  peak 
force  to  display  rapid  ups  and  downs.  However,  these  serial 
effects  were  not  specific  to  any  particular  experimental  condi¬ 
tion.  On  the  whole,  the  statistical  analyses  provided  only  a 
modicum  of  evidence  for  the  presence  of  within-session  feedback 
effects. 
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Chapter  1 
Introduction 


Descriptions  and  theories  of  instrumental  conditioning 
invariably  rest  upon  the  operational  definitions  used  in  obtain¬ 
ing  the  data.  Since  the  appearance  of  Skinner's  definitive  work 
The  Behavior  of  Organisms  (1938),  the  preponderance  of  experi¬ 
ments  investigating  free-operant  behavior  have  defined  the 
response  as  a  momentary  closure  of  a  microswitch  connected  to 
a  spring-loaded  bar  or  lever.  The  principal  datum  of  these 
studies  has  been  the  time  between  switch  closures,  or  rate  of 
responding.  Other  operations  yielding  other  kinds  of  data  are 
possible,  or  even  desirable  as  Jaynes  (1966)  points  out.  This 
paper  reports  on  the  use  of  laboratory  techniques  which  empha¬ 
size  the  quantitative  nature  of  both  response  and  reinforcement, 
and  in  a  manner  which  departs  from  the  customary  analysis  of 
operant  behavior, 

While  investigating  response  differentiation  and  generaliz¬ 
ation,  several  experimenters  (Skinner,  1938;  Hays  &  Woodbury, 
as  reported  by  Hull,  1943;  Arnold,  1945;  and  Trotter,  1956) 
became  mindful  of  the  need  for  a  more  complete  account  of 
operant  behavior  than  that  afforded  by  observing  response  rate. 
But  their  attempts  to  measure  force  of  response  (and  other 
dimensions)  were  hampered  by  inadequate  measuring  devices  and 
by  limited  recording  instruments.  More  recently,  advances  in 
electronics  have  made  available  sophisticated  apparatus  appro- 
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priate  for  measuring  and  recording  the  dimensional  properties 
of  the  bar-press.  Using  such  equipment  Notterman  &  Kintz  (1965) 
began  to  explore  in  detail  several  quantitative  aspects  of 
individual  response  emitted  under  a  variety  of  conditions. 

The  experiments  described  in  this  paper  examined  a  type  of 
reinforcement  contingency  which  Notterman  6  Mints  (1965)  call 
proportional  reinforcement.  Proportional  reinforcement  means 
that  the  delivery  of  a  particular  quantity  of  reinforcement  is 
made  contingent  upon  the  intensity  of  a  selected  response 
dimension.  For  example,  one  of  the  proportional  reinforcement 
"schedules"  used  in  the  present  work  provides  hungry  rats  with' 
sucrose  solutions  which  increase  in  percent  concentration  in 
rough  proportion  to  force  of  a  bar-press  response.  The  pro¬ 
portionality  between . response  dimensions  (whether  force,  duration, 
or  time  integral  of  force)  and  reinforcement  quantity  (whether 
amount,  concentration,  or  some  other  dimension)  need  not  be 
linear.  In  order  to  describe  various  possible  proportional 
reinforcement  schedules,  Notterman  &  Mintz  (1965)  introduced  the 
term  specified  gain.  Specified  gain  (or  just  gain)  is  obtained 
by  dividing  the  reinforcement  quantity  by  the  response  value 
required  to  produce  that  quantity.  Thus,  for  the  example  given 
above,  the  specified  gain  could  be  expressed  in  units  of  percent 
by  weight  of  solute  in  solution/gram  of  force.  (Force  is 
expressed  in  gram  equivalent  weights  for  convenience.  The  force 
of  1.0  gram  at  this  geographic  location  is  given  by  Hodgraan 
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(I960)  as  980.178  dynes.)  These  definitions  of  proportional 
reinforcement  and  gain  make  it  possible  to  formulate  the 
specific  empirical  questions  to  which  the  present  research 
addresses  itself. 

1.  Do  rats  Rfcake  advantage"  of  the  proportional  rein¬ 

forcement  contingency  by  emitting  higher  forces  to  obtain  greater 
quantities  of  reinforcement  they  learn  to  press  harder  to 

get  more  reinforcement  per  response?  And  if  so,  under  what 
circumstances? 

2.  Is  the  specified  gain  of  the  proportional  schedule 
an  important  independent  variable?  That  is,  doss  the  ratio  of 
exertion  to  payoff  influence  force  of  response? 

.3.  Do  the  sequences  of  response  forces  display  any 
important  serial  dependencies  which  might  be  interpreted  as 
indicating  the  presence  of  response" and  reinforcement" induced 
feedback  phenomena?  Can  existing  statistical  methods  be  used 
to  describe  these  supposed  feedback  effects? 

The  number  of  descriptive  terms  which  could  be  applied  to 
proportional  reinforcement  are  probably  as  numerous  as  the 
various  investigators  working  in  the  area  of  conditioning  and 
learning.  Therefore,  a  brief  discussion  of  proportional  rein¬ 
forcement  from  different  points  of  view  serves  to  establish 
parallels  with  related  fields  of  experimentation,  and  thereby 
helps  place' the  present  research  into  a  larger  context. 

In  one  sense,  proportional  reinforcement  defines  a  type 
of  concurrent  schedule  (Catania,  1966) .  That  is,  different 
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force  levels  lead  to  different  quantities  of  reinforcement,  and 
these  multiple  contingencies  are  simultaneously  available  to 
the  organism.  The  responding  animal  can  be  viewed  as  being 
engaged  in  making  choices  based  on  presumably  desirable  rewards 
on  the  one  hand  and  correspondingly  taxing  responses  on  the 
other.  In  the  ordinary  concurrent  schedule  experiment  used  to 
study  choice  behavior,  there  are  two  distinct  manipulanda,  a 
different  reinforcement  schedule  being  operative  on  each. 

However,  when  a  single,-  noiseless,  isometric  manipulandura  is 
used,  as  in  the  present  case,  the  distinctiveness  of  the  multiple 
reinforcement  schedules  may  become  blurred.  With  a  silent, 
isometric  manipulandura  the  organism  must  rely  on  the  cutaneous 
and  kinesthetic  feedback  generated  by  its  responses  in  order  to 

c 

learn  what  kind  of  behavior  is  experimentally  correlated  with  a 
particular  quality  or  quantity  of  reinforcement.  Further,  since 
the  changes  in  behavior  are  to  be  inferred  from  the  intensive 
measures  of.  responding  (as  well  as  from  time  rate  of  responding) , 
the  terminology  associated  with  concurrent  schedules  does  not 
seem  to  be  particularly  appropriate  for  describing  and  analyzing 
the  present  experiments. 

Proportional  reinforcement  can  also  be  cast  in  an  incentive 
framework  (Logan  &  Wagner,  1965} .  However,  most  of  the  data 
that  are  the  underpinnings  of  this  kind  of  theorizing  are 
collected  in  runway  experiments.  As  defined  in  these  experiments, 
a  response  usually  requires  an  appreciable  amount  of  time  for 


execution.  Further,  the  time  between  responses  may  be  as  much 
as  a  minute  or  longer.  Thus  for  the  runway  situation  it  does 
seem  intuitively  appealing  to  conceive  of  the  rat  as  learning 
what  to  "anticipate"  in  the  goal  box.  The  anticipations  or 
anticipatory  goal  responses  become  conditioned  to  successive 
portions  of  the  run-way..  The  reward  conditions  of  the  goal  bdx 
generalize  so  as  to  increase  and  maintain  responding  at  an 
appropriate  level.  On  the  other  han^,  in  the  free-operant 
situation,  "responses''  are  usually  defined  in  such  a  manner  that 
they  occupy  very  brief  periods  of  time  -  intervals  so  brief  as 
to  make  the  ideas  of  incentive  and  anticipatory  bar-pressing 
responses  much  less  appealing. 

Proportional  reinforcement  car.  be  regarded  from  yet  another 
point-of-view.  It  represents  an  experimental  parallel  to 
theoretical  constructs  produced  by  a  "micrcmolar  approach  to 
behavior  theory"  (Logan,  1956) .  Essentially,  this  approach 
recognizes  that  different  intensities  of  the  same  "response" 

(e.g. ,  a  bar  press)  can  actually  be  regarded  as  different 
responses.  Thus  explanations  of  behavior  can  be  made  in  terms 
of  what  quantitative  sub-classes  of  a  particular  "response*’ 
were  either  reinforced  or  not  reinforced  (see  also  Amtonitis, 
1950).  As  formulated  by  Logan,  this  theoretical  approach  relies 
on  many  Hullian  concepts  which  seem  better  adapted  for  explana¬ 
tions  of  behavior  observed  in  a  runway.  Though  the  micromolar 
approach  has  beer,  of  considerable  heuristic  value  (Logan,  1960), 
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its  terminology  will  not  be  adopted  here  because  it  does  not 
appear  particularly  suitable  for  describing  data  generated  in 
the  free-operant  situation. 

Finally,  if  one  views  reinforcement  as  a  stimulus  inde¬ 
pendently  of  its  drive-related  properties  (drive  reducing, 

(Hull,  1943}  or  drive  inducing  (Sheffield,  1966}),  then  pro¬ 
portional  reinforcement  can  be  seen  as  a  special  paradigm  for 
exteroceptive  cueing  of  intensity  of  bar-press  responses 
(Notterman  &  Mintz,  1962)  .  Discriminably  different  values  of 
reinforcement,  serve  as  informative  signals  telling  the  organism 
that  particular  forces  have  been  emitted.  Thus  reinforcement 
associated  with  specific  intensities  could  improve  precision  of 
force  emission  over  and  above  that  attainable  when  discriminably 
different  reinforcements  are  not  correlated  with  response  force. 

Considering  proportional  reinforcement  from  these  varied 
points  of  view  demonstrates  its  relation  to  methods  previously 
used  to  study  choice  behavior  (concurrent  schedules) ,  incentive 
(amount  of  reward  as  it  influences  performance  and  learning) , 
response  differentiation,  and  exteroceptive  cueing  of  response 
force. 

.  Because  the  proportional  reinforcement  contingency  empha¬ 
sizes  the  quantitative  aspects  of  response  and  reward,  some  of 
its  historical  antecedents  also  are  to  be  found  in  the  studies 
dealing  with  amount  of  reinforcement.  A  brief  review  of  this 
work,  follov.’od  by  a  summary  of  experiments  using  proportional 


reinforcement  procedures,  should  set  the  context  for  the  subsequent 
experimental  analysis. 

In  discussing  some  of  the  previous  work  addressed  to  the 
problem  of  instrumental  performance  and  amount  of  reinforcement, 
it  is  convenient  to  distinguish  between  the  two  types  of 
instrumental  learning  tasks  known  as  the  free-operant  and  the 
fixed-operant  techniques.  In  the  free-operant  situation  the 
organism  can  emit  responses  at  any  time.  Thus  the  typical 
apparatus  for  examining  free-operant  behavior  is  the  Skinner 
box.  On  the  other  hand,  behavior  in  the  fixed-operant  situation 
is  paced  by  the  experimenter  so  that  the  response  selected  for 
reinforcement  can  only  be  made  when  the  experimenter  provides 
the  appropriate  stimulating  conditions.  Thus  the  straight 
runway  is  a  representative  device  for  studying  fixed-operant 
behavior.  The  distinction  between  these  learning  tasks  is  made 
here,  not  to  arouse  theoretical  issues,  but  rather  to  suggest 
that  the  two  types  of  procedures  do  not  always  produce  the  same 
kinds  of  relationships  between  performance  and  quantity  of 
reinforcement .  This  conjecture  is  borne  out  in  tine  experiments 
summarized  in  Table  1  {p.  8). 

It  is  clear  from  Table  1  that  a  variety  of  operations  have 
been  used  to  define  amount  of  reinforcement.  For  example, 
amount  (mg.),  volume  (ml.  or  number  of  licks),  access  time 
(scc.),  and  concentration  (%)  are  all  terms  specifying  quantity 
reinforcement.  Furthermore,  these  quantitative  measures  are 
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not  mutually  exclusive:  For  equal  volumes  (ml.)  of  sucrose 
solution,  the  amount  (mg.)  always  covaries  with  concentration 
(%) *  or  for  equal  concentrations  (%) ,  amount  (mg.)  can  covary 
with  access  time  (sec.).  The  designation  is  the  percent 
by  weight  of  the  solute  in  solution  (see  Pfaffman,  Young, 
Dethier,  Richter,  &  Stellar,  1954).  From  the  foregoing,  it  is 
apparent  that  amount  of.  reinforcement  is  not  being  equated  with 
•degree  of  reinforcing  effect"  -  a  dependent  variable. 

Inspection  of  Table  1  suggests  that  a  number  of  general¬ 
izations  may  be  drawn  from  studies  examining  quantity  of  rein¬ 
forcement,  In  the  runway  situation,  it  has  been  demonstrated 
repeatedly  that  running  speed  is  an  increasing  function  of 
amount  (mg.)  of  dry  food,  of  volume  (ml.)  of  sucrose  solution 
or  of  concentration  (%)  of  sucrose  solution  (Grindley,  1929- 
30;  Crespi,  1942;  Zeaman,  1949;  Kraeling,  1961;  and  Smith  & 
Duffy,  1957).  Moreover,  when  the  fixed-operant  is  used,  con¬ 
centration  (%)  (subjectively,  taste)  may  be  relatively  more 
important  in  determining  running  speeds  than  volume  (ml.) 
drunk  (Kraeling,  1961).  The  importance  of  taste  factors  in 
controlling  rate  of  response  is  very  prominent  in  the  free- 
operant  situation  (Guttman,  1953;  Guttraan,  1954;  Collier  £> 
Siskel,  1959;  Collier  &  Myers,  1961):  bar  press  rates  reliably 
increase  as  a  function  of  sucrose  concentration  {%}  up  to 
about  32%,  depending  on  the  schedule  demands.  On  the  other 
band,  with  the  free-oporant  procedure  there  seems  to  be  no 
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clear  relationship  between  access  time  (sec.),  which  covaries 
with  amount  (mg.),  and  rate  of  responding  (Catania,  1963; 
Shettleworth  &  Nevin,  1965).  But  when  the  force  of  the  bar- 
press  response  is  measured  as  a  function  of  amount  (mg.)  of 
reinforcement,  one  finds  that  force  is  a  decreasing  function 
of  amount  (mg.)  (DiLollo,  Ensminger,  &  Nottenrian,  1965; 
Notterman  &  Mintz,  1965).  Collectively,  these  studies  imply 
that  both  amount  (mg.)  and  concentration  (%)  (or  taste  factors) 
reliably  influence  operant  performance,  but  the  nature  of  the 
relationships  depends  on  whether  or  not  the  fixed-  or  free- 
operant  procedure  is  used.  Further,  when  bar-press  behavior 
is  being  examined,  the  relation  between  performance  and  quant¬ 
ity  of  reinforcement  depends  on  whether  or  not  rate  of  respond¬ 
ing  or  force  of  response  is  used  as  the  dependent  variable. 
Listed  in  Table  1  are  six  experiments  which  have  been 
concerned  with  proportional  reinforcement.  Here  again  the 
nature  of  the  results  seems  to  depend  on  what  kind  of  procedure 

is  employed.  Using  the  straight  runway,  Brown  &  Horsfall 

\ 

(1965)  demonstrated  that  rats  proportionally  reinforced  for 
running  speed  ran  raster  than  yoked  controls.  Likewise,  Logan 
(1960)  showed  that  "punishing w  fast  runway  speeds  brought 
about  slower  running  than  that  obtained  with  yoked  controls. 

(It  is  not  entirely  clear  that  this  study  is  really  an  instance 
of  proportional  reinforcement,  though  it  is  certainly  related.) 
A  similar  result  was  obtained  by  Filion,  Fowler,  &  Notterman 
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(1969)  in  the  bar-pressing  situation:  Of  two  groups  of  rats 
receiving  force-correlated  amount  of  food  and  probability  of 
shock,  the  group  getting  the  higher  probability  of  punishment 
displayed  lower  forces  but  continued  to  respond.  In  general, 
however,  attempts  to  get  rats  to  take  advantage  of  proportional 
reinforcement  defined  by  correlating  amount  (mg.)  of  reinforce¬ 
ment  with  intensity  of  Jaar-press  behavior,  have  proved  unsuccess¬ 
ful.  Neither  force  proportional  reinforcement  (mg.  of  food/gram 
of  force)  nor  effort  proportional  reinforcement  (mg.  of  food/ 
gram-sec.  of  effort)  have  produced  evidence  for  the  selective 
reinforcement  of  response  intensity  (Notterman  &  Mintz,*1965; 
and  Filion,  Fowler,  &  Notterman,  1970). 

In  these  latter  two  experiments  the  organism  may  have 
reached  an  upper  limit  of  optional  exertion  which  probably 
depended  on  the  interaction  of  several  variables  (Filion, 

Fowler,  &  Notterman,  1969):  1)  the  absolute  amount  (mg.) 

of  reinforcement  obtained  per  response;  2)  the  specified  gain; 

3)  the  choice  available  of  pressing  again  rather  than  harder; 

4)  the  capacity  of  the  organism  to  discriminate  dhanges  in 
reinforcement  and  changes  in  motor  feedback  cues;  and  5) 
certain  apparatus  limitations  may  have  precluded  the  rats' 
taking  advantage  of  the  proportional  reinforcement.  An 
experimental  examination  of  this  last  possibility  is  described 
in  Chapter  2. 

The  dramatic  way  in  which  rate  of  response  is  influenced 
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by  sucrose  concentration  (see  Table  1)  suggests  that  the  more 

appropriate  procedure  for  obtaining  proportional  reinforcement 

effects'  is  in  terms  of  sweetness  (%),  rather  than  in  terms  of 

amount  (mg.).  If,  as  is  pointed  out  above,  failure  to  obtain  an 

upward  shaping  of  response  force  was  partially  the  result  of 

the  organism’s  inability  to  make  the  necessary  discrimination  in 

amount  (mg.)  of  reinforcement,  then  the  use  of  concentration  to 

« 

define  reinforcement  quantity  should  provide  a  better  opportunity 
for  proportional  reinforcement  to  become  operative.  This 
approach  is  explored  in  Chapter  3. 

The  research  reported  in  Chapter  4  extends  the  procedure^ 
developed  in  Chapter  3,  and  examines  the  importance  of  specified 
gain  as  an  independent  variable. 

In  Chapter  5  detailed  statistical  analyses  are  undertaken. 
This  endeavor  relies  heavily  on  a  type  of  time  series  analysis 
known  as  specral  analysis.  A  discussion  of  this  method  along 
with  a  rationale  for  its  use  is  given  in  Chapter  5.  Suffice  it 
to  say  at  this  point  that  previous  research  (Mints,  1962)  has 

indicated  that  fairly  robust  sequential  effects  do  develop  in 

« 

peak  force  of  response  during  some  types  of  fixed-ratio  respond¬ 
ing.  Further,  one  might  expect  "contrast  effects"  (Reynolds, 
i961)  to  develop  when  the  organism  experiences  changes  in 
amount  (mg.)  or  sweetness  (%)  of  reward  during  a  series  of 
bar-pressing  responses. 
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Chapter  2 

Continuously  Proportional  Reinforcement 

As  noted  earlier,  Notterman  &  Mintz  (1965)  have  reported 
that  rats  do  not  seem  to  take  advantage  of  force  proportional 
reinforcement  by  pressing  harder  to  obtain  more  reinforcement 
per  response.  This  result  may  have  been  dictated  by  certain 
apparatus-related  discontinuities  in  reinforcement  availability. 
The  purpose  of  this  chapter  is  to  describe  an  experimental 
technique  which  removes  the  discontinuities  and,  in  turn, 
provides  the  organism  with  a  continuously  force-proportional 
fcinforcement  schedule. 

Figure  1  shows  some  of  the  values  of  quantity  of  reinforce¬ 
ment  which  wore  actually  used  in  Notterman  &  Mintz’ s  experiment. 
In  Fig.  2  are  depicted  some  of  tho  specified  gain  values  for 
the  same  study.  The  reason  for  the  marked  lack  of  continuity 
In  both  quantity  of  reinforcement  and  specified  gain  lies  in 
the  fact  that  force  was  quantized  into  steps,  and  the  smallest 
Increment  in  reinforcement  per  stop  was  20  mg.  Further,  the 
*w»ximum  number  of  steps  was  limited  to  five,  thereby  setting  an 
u«!pcr  bound  on  quantity  of  reinforcement  per  response  and  also 
treating  a  situation  such  that  above  the  highest  force-rein- 
forceaent  step,  specified  gain  varied  inversely  with  response 
force.  Whether  these  discontinuities  wore  of  sufficient  magni¬ 
tude  tc  nullify  any  tendency  of  the  animal  to  press  harder  is 
'^tenained,  at  least  partially,  by  the  following  factors:  1) 
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Figure  2.  Specified  gain  as  a  function  of  required  peak 
force  for  the  proportional  reinforcement  schedules  given  in 
Fig.  1.  The  labels  on  the  curves  correspond  to  those  in  Fig.  1: 
A  and  B  represent  the  specified  gain  of  the  schedules  used  by 
Notterman  and  Mintz  (1S65)  and  C  and  D  depict  the  specified  gain 
of  the  schedules  used  in  the  present  work. 


t 
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SPECIFIED  GAIN  (mg./gm.) 
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%*ic  force  discrimination  capacities  of  the  organism,  2)  the 
influence  of  quantity  of  available  reinforcement  per  response 
force  level,  and  3)  the  importance  of  specified  gain  (or 
quantity  per  gram  of  force)  in  controlling  behavior.  Relevant 
ic  the  first  issue  are  data  from  the  experiments  of  Notterman 
6  Kintz  (1965)  which  indicate  that — for  the  range  of  values  used 
here — one  cannot  rule  out  the  possibility  that  rats  can  make 
l he  force  discriminations  necessary  for  detecting  the  discontin¬ 
uities,  In  turning  attention  to  the  second  point,  one  can  see 
from  Fig.  1  that  the  actual  reinforcement  contingency  does  not 
provide  more  reinforcement  per  response  when  the  force  is  above 
the  upper  limit.  For  example,  with  the  "increasing  gain" 
schedule,  the  application  of  a  16.  gm.  response  will  produce 
five  pellets,  but  a  24  gflh  response  will  also  yield  five  pellets. 
Thus,  above  the  upper  limit  the  proportional  reinforcement  was 
not  really  operative,  and  no  cue  in  terms  of  quantity  of  rein¬ 
forcement  per  response  was  provided  for  the  responses  over  the 
limit.  With  regard  to  the  third  factor,  one  should  note  in 
fig.  2  that  if  the  specified  gain  is  important  in  determining 
magnitude  of  response  force,  then  the  animal's  experiences  of 
obtaining  a  higher  gain  value  by  pressing  harder  could  have 
keen  counteracted  by  the  experiences  of  pressing  harder  within 
*  given  step  or  above  the  limit  and  receiving  relatively  less. 

P'-t  gram  of  force  exerted.  ; 

The  foregoing  comments  suggest  that  evidence  for  the  differ- 
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cntial  reinforcement  of  higher  response  force  might  be  obtained 
if  the  previous  apparatus-related  discontinuities  in  gain  and 
quantity  of  reinforcement  could  be  circumvented.  The  research 
described  in  this  chapter  sought  to  accomplish  this  by  employ¬ 
ing  a  technique  in  which  access  time  to  a  liquid  reinforcer  is 
taade  directly  proportional  to  force  of  response  by  a  small 
analog  computer  system.  This  stratagem  operationally  limits 
the  discontinuity  and  departure  from  linearity  in  quantity  of 
reinforcement  1)  to  quantization  of  consummatory  activity 
(licks),  2)  to  failure  to  take  full  advantage  of  access  time, 
or  3)  to  the  possibility  that  amount  consumed  is  a  nonlinear 
function  of  access  time.  Provisions  were  made  to  determine 
experimentally  whether  amount  of  licking  and  access  time  are 
linearly  related  within  the  range  of  times  used  in  the  experiment. 

Just  how  a  rat  discriminates  quantity  of  positive  rein¬ 
forcement  has  not  been  explicitly  elaborated.  Certainly  the 
ability  is  at  least  a  function  of  oral  and  post-ingestional 
factors.  It  is  also  possible  (if  not  highly  likely)  that 
•reinforcing  effect"  and  quantity  of  a  particular  reinforcing  agent 
are  not  linearly  related.  The  procedure  described  above  does 
aot  attempt  to  control  for  this  eventuality.  However,  to 
•ssume  at  least  a  monotonic  relationship,  over  a  limited  range, 

between  reinforcing  effect  and  physical  quantity  of  reinforcement 

» 

•ooms  reasonable. 

For  the  continuously  proportional  reinforcement  schedules 


used  in  the  present  experiment,  the  specified  gain  is  held 
constant  for  all  forces  above  the  criterion  (see  Fig.  2). 
Therefore,  if  the  organism  does  have  a  proclivity  to  distribute 
its  forces  so  as  to  maximize  gain,  one  would  not  expect  the 
constant  gain  schedules  to  shape  forces  upward.  (An  examination 
of  the  manner  in  which  variations  in  specified  -gain  influence 
response  force  is  presented  in  Chapter  4.)  On  the  other  hand, 
if  reinforcing  higher  forces  with  larger  quantities  of  reinforce¬ 
ment  does  influence  behavior,  then  an  increase  in  force  of 
response  should  be  observed  during  the  course  of  continuously 
proportional  reinforcement  conditioning. 

Method 

Apparatus 

The  force  recording  apparatus  used  in  this  study  was  a 
modification  of  the  analog  computer  system  described  by 
Notterman  &  Mintz  (1965) .  In  the  present  case,  a  Sanborn 
transducer  (Model  FT A  100-592)  served  as  the  manipulandum. 

The  portion  of  the  transducer  available  for  sensing  forces 
Was  a  13  mm.  diameter  disc  which  protruded  above  its  27  mm. 
diameter  housing  by  1  mm.  The  disc  surface  was  62  mm.  above 
the  grid  floor  of  a  Skinner  box  and  was  positioned  outside  the 
c«ge.  The  manipulandum  was  essentially  isometric,  excursion 
b°ing  iess  than  0.4  mm.  for  forces  up  to  200  gm.  A  Hewlett 
Packard  Digital  Recorder  (Model  561B)  registered  peak  force 
^p)#  duration  (t) ,  and  time  integral  of  force  (/Fdt)  for  all 
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pressures  on  the  disc  exceeding  2.5  gm.  (termed  the  "threshold") , 
which  defined  a  response.  Upon  termination  of  pressures  above 
8.0  gm*  ("criterion")  the  liquid  reinforcement  magazine  was 

operated. 

In  order  to  make  reinforcement  access  time  proportional 
to  response  force,  two  chopper-stabilized  operational  amplifiers 
(Philbrick  Researchers,  Model  K2-XA)  were  required  in  addition 
to  the  analog  computer  system  just  described.  The  first  of 
these,  the  timing  amplifier,  was  connected  to  the  peak  force 
voltage  circuit  of  the  computer  during  a  response,  and  the 
amplifier  output  charged  a  capacitor  with  the  force-equivalent 
voltage.  Immediately  at  the  termination  of  the  response,  the 
timing. amplifier  was  disconnected  from  the  peak  force  circuit, 
and  the  capacitor  charge  fed  back  to  the  input  of  the  timing 
amplifier  along  with  an  adjustable  reference  voltage.  This 
arrangement  resulted  in  a  timing  amplifier  output  which  began 
at  a  negative  value  proportional  to  the  force  of  the  response 
and  decayed  linearly  as  a  function  of  time.  When  this  voltage 
crossed  zero  and  changed  to  positive  sign,  a  diode  ^passed  the 
output  to  the  second  amplifier,  which,  within  .5  sec.  of  zero¬ 
crossing,  operated  a  reset  relay,  thus  ending  the  time  rein¬ 
forcement  period  and  resetting  the  entire  timing  circuit.  The 
reference  voltage  to  the  timing  amplifier  was  adjusted  so  that 
a  one  gram  increase  in  peak  force  above  the  criterion  caused  a 
•5  sec.  increase  in  the  interval  that  the  solenoid-operated 


20 


dipper  (Ralph  Gerbrands  Co.  Model  B-LH)  remained  within  reach 
of  S  in  the  Skinner  box.  Figure  3  presents  a  calibration  curve 
of  the  timing  portion  of  the  apparatus.  The  upper  limit  on  the 
abscissa  of  this  function  was  120  gm.  The  system  employed 
lock-out  relays  in  order  to  prevent  accumulation  of  additional 
access  time  in  case  (1)  S  made  a  criterion  response  during 
dipper  presentation  (these  responses  were  recorded,  however) 
and  so  that  (2)  subcriterion  responses  would  not  deliver  rein¬ 
forcement. 

The  sensing  electrode  of  a  Grason-Stadler  drinkometer 
(Model  E4690A)  was  attached  to  the  dipper  arm.  The  dipper 
volume  (10  ml.)  was  relatively  large,  so  that  the  liquid  level 
would  change  very  little  even  during  prolonged  reinforcements. 
Number  of  licks  during  each  reinforcement  were  recorded  by  a 
U'high  Valley  (Model  1530)  print-out  counter. 

The  Skinner  box  was  located  in  a  sound  insulated  room;  a 
white  noise  generator  on  the  lid  of  the  box  provided  masking 
noise.  To  insure  reliable  operation  of  the  drinkometer,  S 
VAS  momentarily  placed  on  a  mat  of  wet  toweling  befoie  trans¬ 
fer  to  the  box. 

EH^jtcts  and  Procedure 

The  Ss  were  21  male  Wistar  rats  with  mean  weiqht  of  255 
at  the  time  of  receipt  from  Camm  Research  Institute.  Ss 
*°ro  housed  individually  in  a  room  kept  on  a  diurnal  cycle; 
period  of  darkness  was  11:00  P.M.  to  7:00  A.M. 


Figure  3.  Linearity  check  of  apparatus  programming 
access  time  as  a  function  of  peak  force. 
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After  ten  days  of  ad  lib  food  and  water,  water  deprivation 
vat  instituted,  and  over  the  course  of  nine  days,  time  of  access 
to  water  in  the  home  cage  was  reduced  from  1  1/2  hr.  to  1/2  hr. 
During  the  latter  portion  of  these  nine  days,  magazine  approach 
training  was  carried  out.  Each  S  received  approximately  60 
4*scc.  reinforcements  of  water.  On  the  following  four  days, 
each  S  received  two  40-reinforcement  sessions  of  shaping. 

Puring  shaping,  Ss  were  trained  to  reach  out  through  an  opening 
in  the  cage  wall  in  order  to  apply  pressure  on  the  manipulandum. 


This  arrangement  was  designed  to  increase  the  likelihood  of 

developing  uniform  response  topography,  and  to  preclude  the 
♦ 

possibility  of  biting  behavior  being  recorded  as  responses. 

Following  shaping,  two  days  of  continuous  (i.e.,  non¬ 
proportional,  but  regular)  reinforcement  raining  were  admin¬ 
istered  {Phase  I).  Procurement  of  30  4-sec.  tap  water  rein¬ 
forcements  terminated  a  session.  During  Phase  I  the  number  of 
Hcks  per  session  and  the  body  weight  of  each  animal  were  record- 
After  the  two  sessions  of  Phase  I,  Ss  wore  switched  to  a 
food  deprivation  schedule  (water  ad  lib)  in  which* 1  hr.  access 
to  Purina  lab  chow  was  permitted  daily;  Ss  were  maintained  in 
manner  for  the  remainder  of  the  experiment.  The  daily 
**Uon  was  given  1/2-1  hr.  after  S  completed  a  session.  Four 
of  aceomftodation  to  the  food  deprivation  schedule  were 
•1  lowed  before  the  onset  of  Phase  11^.  In  general,  Phase  II 

,uore  time  for  adjustment  to  food  deprivation  was  provided 
^c«usc  only  data  from  the  last  of  20  daily  sessions  was  used  in 
•o  analysis.  Thus  the  data  analyzed  were  based  on  24  days  of 

^lustmcnt. 


involved  sucrose  solution  as  reinforcement;  hence,  the  prior 
steps  of  adjustment  to  both  water  and  food  deprivation.  (See 
Guttman,  1953). 

The  18  Ss  (by  this  time  three  had  died)  were  randomly 
divided  into  three  groups  of  six  each,  which  were  matched  for 
peak  force,  body  weight  and  lick  rate  in  Phase  I.  During  the  20 
sessions  of  Phase  II,  a  continuous  reinforcemei\t  group  (C8) 
received  4-sec.  access  to  an  8%  sucrose  solution  for  each 
criterion  response.  Two  proportional  reinforcement  groups 
received  8%  sucrose  (P8)  and  32%  sucrose  (P32),  respectively. 

For  both  P8  and  P32,  the  amount  of  access  time  (sec.)  earned 
was  one-half  the  magnitude  of  the  force  (gm.)  exerted  for  ^ 

responses  exceeding  criterion.  In  terms  of  access  time  the  gain 
was  0.5  sec. /gram.  The  specified  gain  (in  terms  of  mg. /gram) 
of  these  proportional  reinforcement  schedules  could  only  be 
estimated  from  Ss  lick  volume  and  lick  rate.  Stellar  &  Hill 
(1952)  report  that  there  are  but  small  individual  differences 
in  lick  rates  in  rats,  an  average  value  being  6-7  licks/sec. 

Since  these  Ss  would  be  consuming  solutions  more  viscous  than 
*ater,  measurements  were  taken  on  the  7th  day  of  Phe^se  II  in 
order  to  determine  the  lick  rate  and  lick  volume  for  the  present 
experimental  conditions.  (Amount  of  reinforcement  and  specified 
9ain  are  plotted  as  a  function  of  required  peak  force  in  Fig.  1 
Fig,  2,  respectively.  The  curves  labeled  C  are  for  Group  P32, 
*nd  the  curves  designated  D  are  for  Group  P8.)  Throughout  Phase  II, 
*  session  was  terminated  after  750  licks  had  been  obtained.  S>  was 
Allowed  to  finish  the  last  reinforcement  period  even  if  the  session 
C!>dcd  during  that  reinforcement.  Subject  {33  died  on  the  8th  day 
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0i  phase  II/  reducing  the  number  in  P8  to  5. 

Sucrose  solutions  were  prepared  on  alternate  days  from 
commercial  granulated  sugar  and  tap  water.  The  solutions  were 
•lways  mixed  several  hours  before  use  to  allow  them  to  reach 
room  temperature.  The  percent  specification  is  weight  percent 
of  solute;  i.e. ,  the  8%  solution  was  .371M  and  the  32%  solution 
1.484M. 

Results  and  Discussion 

Measurements  taken  on  the  7th  day  of  Phase  II  revealed 
that  Group  P8  was  licking  at  the  rate  of  4.7  licks/sec.  on  the 
average  and  that  Group  P32  had  a  rate  of  3.7  licks/sec.  Prob¬ 
ably  because  of  differences  in  viscosity,  both  values  are  lower 
thjn  the  6-7  licks/sec.  that  Stellar  &  Hill  (1952)  reported. 
Appropriate  calculations  indicated  that  the  specified  gain  for 
Croup  P8  was  2.8  mg.  sucrose/gra.  of  response  force  (curve  D  in 
ftg.  2)  and  for  Group  P32,  10.4  mg.  suerose/gm.  (curve  C  in  Fig. 
J).  Figure  4  shows  the  average  correlation  of  forces  above 
criterion  with  the  number  of  licks  for  the  associated  rein¬ 
forcement.  Each  point  is  the  group  average  for  a  given  day. 

correlation  coefficients  for  Phase  II  for  individual 
••"dsals  in  the  proportional  reinforcement  groups  ranged  from 
*  low  of  .807  to  a  high  of  .940.  The  data,  therefore, 

I’licate  that  Ss  we.G  in  fact  being  reinforced  with  an  amount 
•rproximately  proportional  to  their  forces.  Moreover,  it 
reasonable  to  assume  that  this  proportionality  was 


«•  Group  »aan  correlation  ooeffScients  Qf  supr 
peak  fox cq  and  nutubor  of  licks  as  a  function  of 

Uuining. 
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virtually  continuous  since  response  force  and  access  time  were 
continuously  related  electrically.  Figure  5  is  a  typical 
session  record  showing  the  consummatory  behavior  during  the 
proportional  reinforcement  intervals. 

Having  established  that  a  continuously  proportional  rein¬ 
forcement  schedule  was  operative  for  Groups  P8  and  P32,  atten¬ 
tion  may  now  be  directed  to  the  main  research  question:  How 
do  the  proportional  contingencies  affect  response  force? 

Table  2  summarizes  the  principal  results  of  the  experiment: 
Neither  of  the  proportionally  reinforced  groups  is  substantially 
higher  in  mean  peak  force  than  the  control  group.  Proportional 
reinforcement  defined  in  terms  of  access  time  does  not  bring 
about  significantly  more  vigorous  bar-pressing.  Either  the 
organisms  were  not  making  the  necessary  discriminations  in 
quantity  of  reinforcement,  or  there  is  no  particular  advantage 
to  their  doing  so. 

The  first  of  these  alternatives  can  be  only  partially 
answered  by  examining  the  possibility  that  changes  in  quantity 
reinforcement,  occasioned  by  variations  in  response  force, 

VCro  insufficient  to  allow  the  organism  to  detect  them.  Figure 
*  is  the  reinforcement  record  for  a  typical  S  in  group  P32 
for  session  20  of  Phase  XI.  The  "spikes"  are  cumulative  records 
of  licking  during  the  single  reinforcement  periods  which  are 
rcpresented.  by  a  downward  deflection  of  the  event  pen.  The 
*vcrage  relative  change  in  access  time,  and  thus  quantity  (mg.) 
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Figure  5.  Cumulative  records  of  number  of  licks  during  each 
reinforcement  interval'  of  session  20  of  Phase  II.  The  period  of 
time  elapsing  between  a  and  b  is  the  inter-reinforcement 
interval.  The  interval  between  b  and  c  is  the  time  that  the 
reinforcement  dipper  was  within  reach  of  the  rat's  tongue. 
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Table  2 

Group  Means  of  Selected  Variables 
for  Session  20  of  Phase  II 


Variable 

Group 

C8 

Group 

P8 

Group 

P32 

Assessment  of  Confi¬ 
dence  Level  Based  on 
Kruskal-Wallace  Test 

mean  peak 
force  (gm. ) 

11.8 

12.7 

12.7 

p  >  .10 

standard  devi¬ 
ation  of  peak 

5.03 

5.57 

6.59 

p  >  .10 

force  (gm. ) 

1 

* 

rate  of-^res- 
ponding 
(R  * s/min. ) 

9.1 

6.2 

5.9 

p  <  .  05 

interresponse 
time  with 
drinking  time 
removed  (sec.) 

3.0 

2.5 

2.7 

p  >  .10 

reinforcement 
rate'1  (mg.  of 

136 

166 

647 

p  <  .  01 

sucrose/min. ) 

Notes:  1.  Both 

P8  and  P32 

:  taken 

separately  are  significantly 

lower 

than  C8, 

p  <  .  05 

;  but  P8 

and  P32  do  not  differ 

from  each  other  p  >  .10.  (These  comparisons  and  the 
ones  below  are  based  on  Wilcoxon's  Rank  Sum  Test.) 

2.  P32  is  higher  than  either  P8  or  C8,  p  <  .05;  however, 

the  difference  between  P8  and  C8  only  approaches 
significance,  .05  <  p  <  .10. 
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of  reinforcement,  can  be  computed  from  these  data.  During  the 
course  of  this  session,  there  were  25  criterion  and  above  respon¬ 
ses;  and,  in  going  from  one  reinforcement  to  the  next,  there  are 
24  possible  changes  in  access  time.  The  mean  value  of  these 
choges  is  0.71,  and  14  of  them  are  greater  than  0.35,  the  value 
that  Tarpy  (1967)  obtained  for  the  Weber  fraction  for  time  dis¬ 
crimination  in  rats.  Thus  if  consummatory  time  provides  a 
sufficient  cue  for  discriminating  quantity  of  reinforcement, 
tuny  opportunities  were  available  for  the  rats  to  experience 
these  differences.  Moreoever,  in  terms  of  milligrams  of  sucrose 
winy  of  the  changes  in  quantity  are  similar  in  magnitude  to 
shifts  in  amount  (mg.  of  dry  pellets)  of  reinforcement  which 
have  been  observed  to  influence  response  force  (Notterman  & 

JUntz,  1965,  Chapter  10).  Despite  the  evidence  that  cues  for 
detecting  variations  in  quantity  of  reinforcement  were  avail¬ 
able,  it  is  still  possible  that  the  close  temporal  proximity 
of  the  various  changes  in  quantity  precluded  their  detection 
because  of  the  rat's  inability  to  form  or  maintain  a  consistent 
standard"  with  which  subsequent  quantities  could  be  Compared. 

Of  course,  other  cues  such  as  caloric  value,  taste,  stomach 
distention,  tonicity,  etc.,  are  also  important  to  the  organism 
in  detecting  changes  in  quantity  of  reinforcement;  however,  the 
Present  data  do  not  permit  an  assessment  of  the  relative  roles 
^  those  various  possible  cues,  although  taste  factors  in 
Articular  arc  examined  in  a  subsequent  experiment  (Chapter  3). 
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The  second  possibility  of  there  being  no  particular  advan¬ 
tage  in  terms  of  efficiency  in  pressing  harder  per  response  may 
l>e  related  to  the  option  available  to  the  rat  of  pressing  again 
rather  than  harder.  Further,  as  previous  research  has  demon- 
atrated  (Filion,  Fowler,  &  Notterman,  1969)  reinforcement  rate 
pay  reflect  the  degree  to  which  each  strategy  is  chosen.  More¬ 
over,  several  experiments  have  shown  that  hungry  animals  tend 
to  meet  the  contingency  of  complex  reinforcement  schedules  by 
maximizing  reinforcement  rate  (Herrnstein,  1961;  Neuringer, 

1967) .  From  Table  2  it  can  be  seen  that  the  reinforcement  rate 
for  both  of  the  proportionally  reinforced  groups  is  higher  than 
that  of  the  control  group.  This  observation  is  in  contrast  to 
the  lower  overall  rate  of  responding  for  these  two  groups  {P8 
and  P32).  The  joint  occurrence  of  low  response  rate  and  high 
reinforcement  rate  for  the  proportionally-^' Enforced  groups  is 
accounted  for  primarily  by  the  fact  that  these  animals  spent 
relatively  more  time  drinking  than  the  controls.  For  example, 
the  emission  of  a  15  gm.  response  by  S  in  Group  P8  would  delay 
the  next  response  for  7.5  sec.  (it  has  already  been  shown  that 
largely  took  advantage  of  access  time);  on  the  other  hand, 

*  gm.  response  made  by  S  in  Group  C8  would  delay  the  next 
response  only  4.0  sec.  The  comparable  magnitudes  of  the  inter- 

response  times  (independent  of  consummatory  time)  for  the  three 

the  proportionally 

groups  suggest  that/romforced  groups  were  bar-pressing  as 
fast  as  was  consistent  with  continuous  consumption  of  each 


reinforcement.  These  two  groups  (P8  and  P32)  obtained  more 
t einforcement/min.  because  of  the  variability  in  response  force 
*»d  because  of  the  gain  of  the  schedule,  not  because  the  Ss  were 
working  "faster"  or  "harder."  This  follows  from  the  fact  that 
there  were  no  reliable  intergroup  differences  in  standard 
deviation  of  peak  force  (see  Table  2).  Therefore,  the  rate 
and  force  of  responding  displayed  by  Group  C8  would  have 
j  induced  reinforcement  rates  similar  to  Group  P8  or  Group  P32 
if  the  control  group  had  been  exposed  to  the  proportional 
•chodulcs.  Thus  the  data  suggest  that  the  option  of  pressing 
i*^rc  often  is  manifested  to  about  the  same  degree  by  the-  three 
groups.  Perhaps  if  an  additional  premium  were  placed  on  press- 
harder,  then  this  alternative  strategy  would  be  selected. 

For  example,  one  could  construct  a  proportional  reinforcement 
tc.icdule  so  that  higher  concentrations  of  sucrose  could  be 

Stained  by  pressing  harder  but  not  by  responding  faster  (sec 
Raptor  3) . 

While  the  results  of  this  experiment  indicate  that  a  force 
»»'>;ortional  reinforcement  contingency  defined  in  terms  of 

drunk  (number  of  licks)  ox*  milligrams  consumed  does  not 
r.rease  probability  of  higher  forces  being  emitted,  they 

'direct  attention  to  the  importance  of  the  animal's  ability 

*  *  £ . 

C*imxnatc  quantity  of  reinforcement  and  its  proclivity 
Press  again  rather  than  more  forcefully.  A  second  experiment, 
‘c*  is  the  subject  of  the  next  chapter,  attempts  to  demon- 


ctrate  the  selective  reinforcement  of  high  response  force  by 
making  concentration  of  sucrose  proportional  to  force  of 
response,  a  tactic  which  provides  a  greater  opportunity  for  dis¬ 
crimination  of  quantity  of  reinforcement  and  thereby  may  increase 
the  likelihood  of  pressing  harder  rather  than  more  often. 
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Chapter  3 

Force-Proportional  Concentration  of  Sucrose  Reward 

Listed  in  Chapter  1  were  five  variables  which  taken  singly 

or  collectively  probably  acted  to  prevent  successful  demonstration 

of  control  of  response  force  by  proportional  reinforcement.  In 

view  of  the  results  described  in  Chapter  2  the  fifth  variable — 

* 

apparatus  limitations  as  they  contributed  to  the  discontinuous 
nature  of  the  contingencies — may  be  safely  eliminated.  The 
four  remaining  variables  are  examined  experimentally  in  this  and 
the  subsequent  chapter. 

By  his  specification  of  proportional  reinforcement  contin¬ 
gency  in  terms  of  concentration  of  sucrose  solution,  and  by  his 
particular  choice  of  the  absolute  quantities  (mg.)  of  sucrose 
involved,  the  investigator  has  sought  to  design  the  present 
experiment  so  as  to  encompass  three  of  these  variables;  1) 
the  absolute  quantity  (mg.)  of  reinforcement  obtained  per  response 
as*  it  influences  force  of  response,  2)  the  choice  available  of 
pressing  again  rather  than  harder;  and  3)  the  capacity  of  the 
organism  to  discriminate  changes  in  quantity  of  reinforcement 
&nt*  changes  in  motor  feedback  cues.  The  role  of  specified  gain, 
tho  fourth  variable,  is  examined  in  Chapter  4.  Summarized  in 
*able  3  are  the  reinforcement  parameters  of  the  present 
Cxpcriment . 

Previous  research  has  demonstrated  a  relationship  between 
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the  reinforcement  quantity  by  the  lower  force  requirement  for  that  quantity. 
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mean  peak  force  and  amount  (mg.)  of  reinforcement  per  response 
(DiLollo,  Ensmir-ger,  &  Notterman,  1965;  Notterman  &  Mintz,  1965). 
These  results  considered  along  with  those  just  reported  suggest 
that  amount  of  reinforcement  per  response  and  mean  peak  force 
can  be  expected  to  covary  within  a  limited  range.  This  general¬ 
ization  poses  difficulties,  however,  for  predicting  the  outcome 
of  proportional  reinforcement  operations.  Specifically,  for 
a  given  peak  force  distribution,  a  proportional  reinforcement 
contingency  makes  available  much  more  reinforcement  per  response 
than  its  non-proportional  counterpart  -  regular  (or  continuous) 
reinforcement.  Thus,  although  one  might  intuitively  expect 
proportional  reinforcement  to  select  high  forces,  at  the  same 
time  one  might  anticipate  that  the  amount  of  reinforcement  ob¬ 
tained  per  response  would  thereby  increase,  perhaps  reciprocally 
tending  to  decrease  mean  peak  force.  Furthermore,  if  sweetness 
as  well  as  amount  is  made  proportional  to  peak  force,  then 
the  force  decreasing  effects  of  greater  amounts  might  be  en¬ 
hanced,  because  not  only  would  the  amount  increase  but  also  the 
taste  stimulation.  Thus  proportional  reinforcement  prpcedures 
Using  positive  reinforcers  may  be  inherently  self-defeating. 

For, these  reasons  the  importance  of  the  values  and  range  used 
*0r  amount  of  reinforcement  per  response  becomes  crucial. 

The  critical  role  of  amount  (mg.)  per  response  is  mani- 
*csted  in  at  least  two  ways.  First,  suppose  two  groups  of 
r«ts  are  exposed  to  the  same  proportional  reinforcement  contin- 
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gency  in  terras  of  amount  (mg.)  available  per  response  (Groups 
3  and  4  in  Table  3) .  Further,  suppose  that  Group  3  receives 
taste  cues  correlated  with  the  peak  force  requirements.  Under 
these  conditions  one  can  presume  higher  peak  forces  on  the  part 
of  Group  3  as  evidence  that  these  subjects  pressed  harder  to 
obtain  the  sweeter  solutions.  On  the  other  hand,  should  Group 
3  prove  to  be  lower  than  or  equal  to  Group  4,  one  could  conclude 
that  the  taste  stimulation  has  little  additional  effect  on  peak 
force.  This  latter  outcome  would  also  suggest  that  proportional 
reinforcement  acts  only  to  increase  the  amount  of  reinforcement 
per  response  which  in  turn  reduces  mean  peak  force.  Comparison 
of  mean  F  for  these  groups  should  provide  some  decision  on 
this  issue.  If,  in  a  proportional  reinforcement  situation,  the 
•mount  obtained  per  . response  really  represents  an  equilibrium 
point  between  the  tendency  to  press  harder  to  get  more  versus 
the  opposing  tendency  for  amount  obtained  per  response  to 
decrease  force,  then  mean  peak  force  and  amount  obtained  per 
Response  should  be  highly  correlated.  Therefore,  for  any  two 
groups  sharing  a  common  lower  criterion  and  receiving  s'imilar 

(mg.)  per  response  (as  a  consequence  of  their  behavior), 
force  levels  should  be  similar.  However,  should  one  of  the 
groups  x-eceive  a  force-correlated  taste  cue,  and  should  this 
display  the  higher  peak  force,  then  such  a  finding  would 
i  ^  ^idence  of  the  efficacy  of  taste  proportionality  in  estat 
I  *Mng  control  of  peak  force.  For  example,  Grov'n  2  shares 
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lower  limit  criterion  of  8.0  gm.  with  Group  4.  But  for  a  given 
distribution  of  emitted'  peak  forct,  more  reinforcement  is 
available  for  Group  4  than  for  Group  2.  Therefore,  if  Group 
2  comes  to  settle  for  a  level  of  amount  of  reinforcement  per 
response  roughly  equivalent  to  that  of  Croup  4,  then  such  behavior 
can  be  interpreted  as  demonstrating  the  existence  of  the  tend¬ 
ency  fer  rats  to  press  harder  to  get  sweeter  reinforcements. 

From  a  parametric  point  of  view  the  absolute  quantity  of 
reinforcement  available  per  response  may  have  important  impli¬ 
cations  for  proportional  reinforcement  experiments.  That  is, 
the  use  of  either  greater  or  smaller  quantities  than  these, 
previously  tried  might  have  produced  evidence  for  the  selective 

•haping  of  high  response  force.  In  previous  studies,  the 

the 

•mallest  amount  (mg.)  as  well  as  /smallest  increment  in  rein¬ 
forcement  was  a  20  mg.  food  pellet.  In  Chapter  2  of  this  paper, 

22  mg.  (dry  weight  of  sucrose  actually  in  solution)  was  the 
Smallest  amount  used.  These  quantities  may  have  been  so  large 
to  render  any  tendency  to  press  harder  essentially  unimportant 
compared  to  the  tendencies  to  conserve  energy  and  to  maintain 
*  satisfactory  reinforcement  rate.  Inspection  of  Table  3 
reveals  that  6.35  mg.  of  sucrose  is  the  smallest  amount  available 
Per  response  in  the  present  experiment.  Group  1  and  Group  4 
*crvo  as  anchor  points  for  assessing  the  degree  to  which 
Proportional  reinforcement  may  control  peak  force  independently 
^  taste  when  very  small  amounts  are  used.  Using  such  small 


38 


quantities  has  the  advantage  of  reducing  possible  effects  of 
satiation,  which  may  be  the  underlying  basis  of  the  previously 
observed  relationship  between  mean  peak  force  and  amount  cf 
reinforcement  (cf.  DiLollo,  Ensminger,  &  Notterman,  1965). 

Defining  proportional  reinforcement  in  terms  of  concentration 

of  sucrose  should  place  a  premium  on  pressing  harder — rather 

than  more  often — and  thus  should  provide  some  degree  of  control 

over  any  proclivity  to  favor  rate  over  force.  This  follows 

from  the  fact  that  the  sweeter  solutions  can  be  obtained  only 

by  relatively  greater  force  -  rate  of  response  alone  cannot 

contribute  to  the  procurement  of  the  sweeter  reinforcements. 

Nevertheless,  rate  of  resonse  can  still  affect  the  overall 

rate  at  which  reinforcement  is  obtained.  For  these  reasons  the 

present  experiment  should  provide  more  information  on  the  v:ay 

force  and  rate  interact  to  influence  reinforcement  rate. 

,  Consideration  of  the  discriminative  capacities  of  the 

6f9anism  encourages  the  specification  of  proportional  reinforce- 

I  **cnt  schedules  in  a  discontinuous  manner  -  a  procedure  criti- 

I  «Ucd  in  Chapter  2,  In  fact,  inherent  in  the  2-step  contingency 

h  Group. 2  (see  Table  3)  is  a  degree  of  discontinuity  which 

|  **  exceeds  that  found  in  the  earlier  proportional  reinforce- 

H  experiments  (Notterman  &  Mints,  1965;  Filion,  Fowler,  & 

■  1959),  though  it  is  consonant  with  techniques  used 

IP  l!°^dry  (1962).  Despite  the  previous  criticisms,  two  consider 
Brl^lnn 

HI  *  s  recommend  this  procedure.  First,  the  results  reported 
^ftpter  2  imply  that  the  discontinuities  were  not  import. 


. 
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Prevdnting  hhe  upward  shaping  of  response,  force  in  previous 
research,  because  removal  of  the  discontinuities  had  virtually 

<A 

no  effect  on  peak  force.  Second,  the  discontinuities  in  amount 
of  reinforcement  and  in  the  correlated  force  requirement  may 
actually  be  essential  to  the  organism's  discrimination  of  changes 
in  quantity  of  reinforcement  and  changes  in  motor  feedback  cues. 
Thus,  the  proportional  reinforcement  studies  cited  above  did 
not  obtain  positive  results  perhaps  because  the  operations  did 
not  include  sufficiently  distinguishable  steps.  The  concentra¬ 
tions  of  sucrose  and  the  force  requirements  for  Group  2  were 
selected  with  a  view  toward  assuring  that  the  two  values  on 
each  continuum  would  be  easily  discriminable  from  one  another. 

The  Weber  ratio  for  sucrose  solutions  in  hungry  rats  is  prob¬ 
ably  between  .25  and  .50  for  the  range  of  values  used  in  this 
experiment  (Campbell,  1958).  Also  24%  is  near  the  value  which 
produced  the  maximum  rate  of  response  in  Guttman's  (1954) 
study.  Therefore,  rats  exposed  to  the  Group  2  contingency  should 
*aSily  discriminate  the  changes  in  quantity  of  reinforcement 
since  a  200%  change  is  involved.  No  clear-cut  psychophysical 
data  on  force  discrimination  in  rats  are  available?  therefore, 
tho  chorees  of  the  force  requirements  were  somewhat  arbitrary, 
though  they  were  selected  so  as  to  be  comparable  to  values  used 
fruitfully  in  previous  research  efforts. 

In  choosing  the  force  requirements,  it  is  important  to 
nsider  whether  the  rats  themselves  emit  a  sufficiently  broad 
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range  of  forces  during  the  initial  sessions  of  training  as  to 

assure  that  all  the  force-reinforcement  contingencies  of  a 

proportional  schedule  are  sampled.  To  increase  the  likelihood  of 

this  happening,  the  bar-press  response  was  reinforced  during  a 

preliminary  phase  without  regard  to  actual  force  emitted. 

Reinforcement  delivery  was  based  only  on  response  topography;  i.e 

paw-pressing.  If  it  is  assumed  that  this  procedure  initially 

reinforces  a  sufficiently  broad  range  of  forces,  it  must  also 

be  assumed  that  the  "random"  shaping  might  produce  initial  mean 

F  values  v/hich  differ  widely  from  one  another.  This  latter 
P 

observation  has  two  important  implications  for  evaluating  the 

mean  data.  First,  the  changes  which  individual  animals 

display  during  the  course  of  conditioning  should  be  interpreted 

in  the  light  of  both  their  initial  F  distributions  and  the 

P 

particular  force  requirement  to  which  they  were  exposed.  For 
example,  in  order  to  provide  evidence  of  responsiveness  to 
proportional  reinforcement,  an  animal  in  Group  2  with  an  ini¬ 
tially  high  mean  F^  need  not  display  an  increase  in  mean  F^. 

is  sufficient  for  the  animal  to  maintain  its  high  'level  of 
response  force.  On  the  other  hand,  an  animal  in  Group  2  which 

begins  at  a  low  mean  F  ,  but  remains  there,  is  probably  not 

P 

responding  to  the  force-correlated  sweetness  of  the  schedule. 

*  fiocond  implication  of  the  shaping  procedure  is  that  statistical 
c°mparisons  between  the  independent  groups  should  be  made  only 
*ftcr  extended  conditioning,  when  the  influence  of  the  pre¬ 
liminary  shaping  has  become  minimal. 
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In  view  af  the  foregoing  rationale  the  present  experiment  is 
ceen  as  addressing  itself  to  these  specific  questions:  1) 

When  sweetness  of  reward  is  correlated  with  force  requirement 
do  rats  press  harder  to  obtain  the  sweeter  solutions?  2) 
independently  of  taste  factors,  do  proportional  reinforcement 
schedules  based  on  very  small  quantities  of  reinforcement  lead 
to  the  emission  of  high  force?  {That  is,  were  previous  failures 
to  demonstrate  the  supposed  effect  merely  the  result  of  an 
unfortunate  choice  of  parameters?)  3)  How  do  force  and  rate  of 
response  interact  in  determining  the  overall  rate  of  rein¬ 
forcement? 

Method 

Apparatus 

The  response  measuring  and  recording  equipment  was  iden¬ 
tical  to  the  apparatus  described  in  Chapter  2.  However,  the 
delivery  of  sucrose  solutions  was  carried  out  in  an  entirely 
different  manner.  Five  liquid  solenoid  valves  (Lehigh  Valley 
Electronics,  Model  1527)  were  individually  connected  by  means 
1/4  tygon  plastic  tubing  to  a  special  food  cup  made  of 
^chined  brass  (see  Fig.  6).  Each  of  the  five  reservoirs  connected 
to  the  valves  contained  a  different  concentration  of  sucrose 
lution.  Calibration  of  each  of  the  valves  was  accomplished 
manual  adjustment  of  the  flow-rate  needle -valve  and  by  fine 
^Justmcnts  of  the  pulse  width  which  operated  the  solenoid 

Since  the  contingencies  for  Group  4  involved  only  one 
^central-. ion,  the  volume  of  liquid  dispensed  by  the  8%  valve 
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after  each  effective  response  was  controlled  by  five  interval 
timers.  Various  electrical  gates,  pulse  formers,  relays,  etc. 
were  used  to  select  the  appropriate  timer  corresponding’ to 
the  magnitude  of  force  exerted  on  the  manipulandum. 

Inasmuch  as  the  solenoid  valves  operated  with  a  minimum  of 
noise,  a  pellet  dispenser  was  wired  to  produce  a  single  click 
each  time  a  supra-criterion  response  was  terminated.  Because 
of  the  large  number  of  "and"  gates  and  relays  used  in  programming 
the  reinforcement,  there  was  an  interval  of  about  .6  sec. 
after  termination  of  such  responses  during  which  the  reinforce¬ 
ment  equipment  was  unresponsive  to  subsequent  responses.  Further, 
measurement  and  recording  of  pressures  on  the  manipulandum  did 
not  occur  during  the  first  .25  to  .30  sec.  of  this  interval. 
Subjects  and  Procedure 

The  subjects  were  32  male  Sprague-Dawley  rats  supplied  by 
Charles  River  Breeding  Laboratories.  The  animals  wore  housed 
individually.  Five  days  after  receipt,  the  animals  were 
placed  on  a  23-hr.  hunger  rhythm  which  provided  l~hr.  access 

to  Purina  lab  chow  once  per  day.  Water  was  continuously 

<  J 

available  in  the  homo  cage.  Twenty-three  days  of  this  regime 
proceeded  initiation  of  the  experiment  at  which  time  the  mean 
tody  weight  was  311  gm.  A  daily  period  of  darkness  from  11:00 
to  7:00  A.M.  was  provided  in  the  vivarium. 

Magazine  training  and  shaping  (Phase  I)  were  carried  out 
ln  the  manner  described  in  Chapter  2,  except  reinforcements 
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always  consisted  of  .05  ml.  of  8%  sucrose  solution  during  this 
phase.  During  shaping,  reinforcement  was  delivered  only  on  the 
basis  of  response  topography;  the  force  of  response  was  not 
taken  into  account.  This  procedure  was  used  to  insure  that  Ss 
would  emit  a  broad  range  of  peak  forces  during  the  initial 
session  (s)  of  acquisition,  thereby  increasing  the  likelihood 
that  all  the  force  requirements  and  their  associated  reinforce¬ 
ments  would  be  sampled.  Two  subjects  were  discarded  during  Phase 
I  because  of  their  extremely  aberrant  response  topographies. 

The  remaining  30  subjects  were  randomly  assigned  to  the  four 
groups  listed  in  Table  3.  Acquisition  (Phase  II)  began  imme¬ 
diately  after  Phase  I  and  continued  for  15  daily  sessions. 

Each  session  was  terminated  after  procurement  of  50  reinforce¬ 
ments  regardless  of  the  number  of  reinforcements  obtained  at 
each  force  level.  Subjects  were  run  in  a  counter-balanced 
order  so  that  one  subject  from  Group  3  v/as  followed  by  a 
subject  from  Group  2  which  in  turn  was  followed  by  a  member  of 
Croup  1,  and  so  on.  The  same  order  was  maintained  throughout 
fhasc  II.  During  Phase  II  the  subjects  were  maintained  on  a 
hunger  rhythm  which  provided  one  hour  access  to  mash 

Tj'roximately  one  hour  after  a  session. 

* 

Sucrose  solutions  were  prepared  from  tap  water  and  commer- 
'•*1  sugar  every  three  or  four  days.  The  entire  reinforcement 
Tensing  -apparatus,  including  tubing  and  valves,  was  washed 
roughly  before  the  fresh  solutions  were  placed  in  the  reser- 
8«  and  the  solenoid  valves  were  recalibrated. 
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Results  and  Discussion 

Since  evidence  that  sweetness  of  proportional  reinforcement 
influences  response  force  may  be  available  from  the  performance 
of  individual  animals,  as  well  as  from  intergroup  comparisons 
of  mean  and  other  variables,  the  analysis  which  follows 
examines  both  sources  of  information. 

Table  4  presents  the  mean  and  standard  deviation  of 

for  each  of  the  subjects  for  the  first  and  last  three  sessions 

of  Phase  II.  The  last  three  sessions  were  combined  because 

mean  peak  force  had  ceased  to  change  over  these  sessions. 

Choosing  the  last  three  sessions  as  being  representative  of 

stable  performance  suggested  the  further  choice  of  the  first 

provide  a 

three  sessions  so  as  to /comparable  number  of  observations  in  the 
initial  and  final  distributions.  Table  4  also  gives  the  results 
of  t  tests  of  the  differences  in  the  initial  and  final  means 
(see  note  1,  Table  4).  It  is  clear  that  there  are  substantial 
individual  differences  in  the  way  the  subjects  change  during 
the  course  of  Phase  FI  conditioning.  However,  within  each  of 
ihe  groups  there  appears  to  be  some  consistency  in  that  the 
changes  in  the  mean  generally  reflect  the  force  requirements 
imposed,  except  possibly  for  Group  4.  For  example,  in  Group  1, 
*lve  subjects  display  a  significant  change  -  four  are  decreases 
*nd  one  an  increase.  The  one  subject  which  increases  was  the 
lowest  in  the  group  during  the  initial  sessions.  Thus,  the 
^crease  for  this  animal  may  merely  be  a  reflection  of  the 
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•  Table  4 

^o-c  parameters  of  the  Individual  Animal  Force  Distributions 
for  the  Initial  and  Final  Sessions  of  Phase  II 


Initial  Final 


Croun 

Subiect 

Mean 

S.D. 

N 

Mean 

S.D. 

N 

t 

3 

22.4 

18.1 

227 

T47T 

7.8 

191 

-6.199**** 

9 

13.9 

11.6 

226 

12.8 

6.8 

202 

-1.254 

16 

21.2 

12.7 

191 

14.0 

7.4 

184 

-6.739**** 

1 

17 

30.0 

20.1 

186 

21.9 

10.0 

167 

-4.809*** 

21 

26.0 

18.0 

209 

19.8 

13.6 

196 

-3.908*** 

29 

11.4 

7.5 

258 

13.6 

9.7 

205 

2.667** 

35 

13.7 

6.4 

180 

14.2 

7.8 

194 

0.625 

Mean 

19718 

nm 

5 

22.0 

•16.0 

193 

19.3 

14.3 

198 

-1.728 

11 

14.6 

11.0 

229 

32.5 

16.8 

166 

11.978**** 

13 

17.7 

9.4 

186 

16.3 

7.4 

181 

-1.569 

2 

14 

15.8 

13.6 

236 

28.2 

32.2 

233 

5.406*** 

15 

20.1 

13.6 

205 

24.0 

20.8 

225 

2.311* 

23 

26.1 

22.3 

189 

30.3 

23.6 

155 

1.706 

30 

23.1 

19.9 

201 

27.7 

33.5 

237 

1.795 

33 

11.1 

8.6 

268 

11.1 

4.2 

187 

-0.098 

Mean 

18.8 

23.7 

1 

12.4 

10.7 

292 

27.3 

20.0 

185 

9.355**** 

2 

19.5 

14.4 

201 

18.4 

14.7 

211 

-0.737 

6 

• 

27.0 

19.0 

185 

25.3 

14.5 

234 

-1.016 

8 

12.6 

8.1 

218 

8.3 

3.5 

269 

-7.258**** 

3 

20 

15.2 

10.9 

207 

20.2 

13.3 

194 

4.128*** 

28 

11.1 

9.2 

289 

17.2 

7.8 

176 

7.743**** 

'  36 

13.3 

8.1 

203  - 

19.1 

8.5 

161 

6.530**** 

38 

12.,  4 

9.4 

254 

23.4 

19.6 

20£ 

7.434**** 

Mean 

157? 

19.9 

4 

15.4 

11.0 

249 

19.9 

19.3 

225 

3.089** 

10 

15.8 

8.6 

186 

11.1 

4.9 

199 

-6.450**** 

18 

11.6 

11.1 

290 

12.9 

11.0 

304 

1.408 

4 

25 

16.7 

10.7 

209 

19.8 

14.4 

244 

2.597** 

26 

11.4 

9.3 

282 

20.4 

19.5 

234 

6.441**** 

31 

15.0 

9.2 

202 

14.5 

12.0 

230 

-0.479 

34 

15.3 

10.6 

224 

13.4 

10.  3 

266 

-1.932 

Mean 

m 

IM 

^ '  '.i'll;  1 

}*  .  The  individual  animal  distributions  are  clearly  non-normal, 
^:,viously,  they  are  not  homosccdastic.  The  distributions  are, 

unimodal.  The  t  tests  were  based  on  the  raw  data.  As  shown 
, ••M'tor  5  a  log,Q  transformation  does  bring  these  distributions 
*(,  /  normal.  Nevertheless ,  statistical  tests  based  on  the  login 

t  °rmc*d  data  do  not  substantially  alter  the  outcome.  Thus,  the1 
vcrc  used  here  to  give  the  reader  a  firmer  feel  for  the 
quantities  involved. 

•  means  a  decrease  during  training;  *p  <  .05,  **p  <  .01, 

•  <  .0002,  ****p  <  .000001. 


47 


initial  sub-normal  level  of  responding.  The  general  tendency 

for  Group  1  to  decline  in  mean  P  as  a  function  of  training  (see 

P 

rig.  7).  i-s  consistent  with  the  findings  of  Notterman  6.  Mintz 
(1965),.  although  the  rate  of  change  is  much  slower  in  the  present 


case.  The  significant  changes  in  for  the  subjects  in  Groups 


j  and  3„  in.  all  cases  but  one  (#8) ,  are  compatible  with  the 
expectations  outlined  ea,rlier.  If  the  force  proportional  sweet¬ 
ness  did  act  to  condition  the  emission  of  high  response  force, 
then  subjects  in  Groups  2  and  3  should  show  increases  in 
daring  conditioning  or  they  should  display  no  significant  change. 
The  single  exception,  §8,  which  decreases  significantly  in  mean 


r  ,  invites  a  variety  of  conjectures:  It  may  be  an  indication 


that  this  subject  was  deficient  in  gustatory  sensitivity. 

Or,  equally  plausible  is  the  possibility  that  some  rats  find 
it  advantageous  to  press  harder  to  get  sweeter  tastes  and  others 
-o  not.  For  Group  4,  the  changes  in  F^  do  not  present  as  clear 

*  picture  as  the  other  groups.  Whereas  the  performance  of  §26 

suggest  an  adventitious  rise  in  peak  force  because  of  the 
•-Eject's  initially  sub-normal  level,  the  increases  of  §4 
*r-d  §25  perhaps  indicate  a  real  tendency  to  press  harder  to  get 
r°r.c  of  the  same  solution.  This  possibility  is  examined  more 
closely  in  the  following  analysis.  (The  possible  effects  of 
*i'>cified  gain  arc  reserved  for  consideration  in  Chapter  4.) 
Examination  of  the  individual  subjects'  data  does  not 

*  ford  firm  decisions  on  the  questions  raised  earlier.  For 


I 


|  Figure  7.  Mean  F^  as  a  function  of  Phase  II  training  for 

l|  the  four  experimental  groups.  Each  data  point  is  the  group  mean 
of  individual  subject  means  for  &  block  of  three  sessions. 
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this  reason ,  a  one-way  analysis  of  variance  using  the  method 
of  planned  comparison  (Hays,  1963,  pp.  460-471)  was  conducted 
on  several  variables  for  the  final  sessions  of  Phase  II.  Table 
5  summarizes  the  questions  and  their  probable  answers.  The 
computations  were  made  with  and  without  subject  §8  in  order  to 
see  how  inclusion  or  exclusion  of  this  possibly  aberrant 
subject  influences  the  confidence  of  the  decision.  (Subject 
18  was  included  in  the  plots  of  the  group  data  which  are  pre¬ 
sented  in  Figs.  7,  8,  9,  and  10  for  mean  F  ,  amount  of  rein- 

P 

forcement/response,  reinforcement  rate,  and  rate  of  response 
respectively.)  The  statistical  evaluation  of  mean  peak  force 
does  suggest  that,  on  the  average,  rats  will  emit  higher  forces 
to  obtain  sweeter  tasting  solutions  (affirmative  answers  to  2, 

4,  and  5) .  This  conclusion  is  buttressed  by  the  results  ob¬ 
tained  for  amount  (mg.)  of  reinforceroent/response.  It  is  clear 
from  Table  5  and  Fig.  8  that  Group  3  displays  an  increasingly 
(by  sessions)  higher  amount  obtained  per  response  than  Group 
4,  and  that  Group  2  is  almost  identical  to  Group  4  in  this 
regard.  Thus  the  high  peak  forces  emitted  by  the  subjects 
reposed  to  the  sweetness  proportional  schedules  cannot  be 
attributed  to  lower  amounts  of  reinforcement  per  response. 

specifically,  in  terms  of  amount  of  reinforcement  available 
response.  Group  3  and  Group  4  are  identical;  nevertheless, 
tne  loss  conservative  answer  to  question  4  of  Tabl©  5  is 
flowed)  Group  3  pressed  significantly  harder  presumably  because 


utn?-  ar<>J  nul^  X?i7-.- .  :£ti  f#7  71 
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Table  5 


intergroup  Comparisons  by  the  Method  of  Planned  Comparisons  for 
Selected  Variables  for  Final  Three  Sessions  Combined  of  Phase  II 


Mean  F 

P 

1  Did  reinforcement  operations  differ- 
entially  influence  mean  F  ? 

(overall  F  test)  P 

2  Are  Groups  2  and  3  together  differ¬ 
ent  from  Group  4  in  mean  F  ? 

P 

y  Is  Group  4  higher  in  mean  F  than 

.  Group  1?  p 

4  Is  Group  3  higher  in  F  than 
Group  4?  P 

$  Is  Group  2  higher  in  mean  F  than 
Group  4?  P 


fc  Did  the  reinforcement  operations 
differentially  influence  reinforce¬ 
ment/response?  (F  test) 

1  Does  Group  3  differ  from  Group  4 
in  reinforcement/response? 

I  Does  Group  2  differ  from  Group  4 
in  reinforcement/response? 


*  Did  reinforcement  operations 

influence  differentially  reinforcement 
rate?  (f  test) 

*v  Does  Group  3  differ  from  Group  4 
in  reinforcement  rate? 

*1  Does  Group  2  differ  from  Group  4 
in  reinforcement  rate? 

Rate  of  Responding 

^  rein^orceraeni-  operations 

differentially  influence  response 
rate?  (p  test) 


With  #8 


Without 


Yes,  p  <  .05 

Yes,  p 

<  .05 

Yes,  p  <  .05 

•yes,  p 

<  .01 

No,  p  >  .10 

No,  p 

>  .10 

No,  p  >  .10 

Yes,  p 

<  .05 

Yes,  p  <  .01 

Yes,  p 

<  .005 

Response  (mg/R) 
Yes,  p  <  .01 

Yes,  p 

<  .01 

yes,  p  <  .05 

Yes,  p 

<  .02 

No,  p  >  .10 

No,  p 

>  .10 

(mg/min) 

Yes,  p  <  .01 

Yes,  p 

<  .01 

Yes,  p  <  .05 

Yes,  p 

<  .02 

No,  p  >  .IQ 

No,  p 

>  .10 

No,  p  >  .10 

No,  p 

>  .10 

i 

< 

% 

\'4&f 


&=: 


if 


51 


r .  £*£$ .'. 


Figure.  8.  Average  amount  of  sucrose  (in  milligrams) 
obtained  per  response  as  a  function  of  Phase  II  conditioning. 
Each  data  point  is  the  group  mean  of  individual  subject  jneans 
for  three  sess5.ons.  That  is,  the  total  amount  obtained  in  a 
session  was  divided  by  the  number  of  responses  emitted  during 
that  session.  The  mean  of  three  sessions  was  obtained  for  a 
given  individual.  Then  these  values  were  combined  to  obtain 
a  group  mean. 
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Figure  9.  Average  reinforcement  rate  as  a  function  of 
Phase  II  conditioning  for  the  four  experimental  groups. 

Each  point  is  a  group  mean  of  individual  subject  means  computed 
over  three  sessions.  The  reinforcement  rate  for  a  single 
subject  was  obtained  by  dividing  the  amount  of  sucrose  (in 
milligrams)  procured  in  a  session  by  the  session  time  (measured 
to  the  nearest  .016  min.). 


AMOUNT  OF  REINFORCEMENT 
PER  MINUTE  (  mq./min.) 


Figure  10.  Average  rate  of  responding  as  a  function  of 
Phase  II  training.  Each  point  is  a  group  mean  of  individual 
subject  means  computed  for  three  sessions.  Rate  of  responding 
was  obtained  by  dividing  the  number  of  responses  emitted  in  a 
session  by  the  session  time. 
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of  the  correlation  of  greater  sweetness  with  higher  peak  force 
of  response. 

It  might  be  argued  that  the  high  forces  of  Group  2  and 
Group  3  are  a  manifestation  of  "drive  induction"  (Sheffield, 
1966) .  This  interpretation  seems  unlikely  in  view  of  the 
results  presented  in  Chapter  2.  It  will  be  recalled  that  two 


^dependent  groups  of  rats  separately  received  two  different 

concentrations  of  sucrose  reinforcement.  But  there  were 

essentially  no  differences  between  the  groups  -  neither  in  mean 

peak  force  nor  in  rate  of  responding. 

The' similarity  in  mean  F  for  Group  1  and  Group  4  (non- 

P 

rejection  of  null  hypothesis,  item  3,  table  5)  is  in  accord  with 

the  previous  investigations  which,  have  attempted,  and  failed, 

i  to  demonstrate  the  upward  shaping  of  response  intensity 

(either  F  or  time  integral  of  force)  by  use  of  quantity  of 
P 

reinforcement.  The  present  results  show  that  even  when  very 

small  absolute  quantities  are  the  basis  of  the  proportional 

reinforcement  schedule,  rats,  on  the  average,  do  not  press 

harder  to  obtain  more  of  the  same  substance.  It  is  lbgically 

clear  that  this  negative  conclusion  should  be  limited  to  the 

parametric  conditions  represented  by  previously  used  continuous 

and  five-step  proportional  schedules.  Other  quantitative 

arrangements  could,  of  course,  produce  different  results. 
Reinforcement  rate  was  significantly  influence  by  the 

reinforcement  schedules  (item  9,  Table  5).  This  result  is  to 

be  expected  in  viev;  of  the  joint  occurrence  of  high  .forces  for 


Group  2  and  Group  3,  and  the  lack  of  reliable  differences  in 
average  rate  of  responding  among  the  four  groups.  Thus  the 
significant  intergroup  differences  in  reinforcement  obtained  per 
response  and  in  reinforcement  rate  are  accounted  for  primarily 
by  the  more  forceful  bar  pressing  of  the  sweetness  proportional 
groups.  As  expected,  mean  and  amount  of  reinforcement  per 
response  are  highly  correlated  for  Group  II  (r  =  .741,  p  <  .05) 
and  Group  III  (r  =  .800,  p  <  .05). 

As  an  overall  conclusion  it  may  be  stated  that  the  cueing 
of  response  force  with  concentraton  of  sucrose  results  in  more 
forceful'  responding.  On  the  other  hand,  when  increased  amounts 
(without  the  taste  cue)  are  associated  with  higher  forces  the 
rats,  on  the  average,  do  not  press  harder.  However,  both 
experimental  procedures  lead  to  considerable  individual  differ¬ 
ences,  so  that  both  types  of  groups  include  subjects  which 
seemingly  do.  press  harder  and  subjects  which  do  not  differ 
from  regular  reinforcement  control  animals. 
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Chapter  4 

Some  Effects  of  Gain  on  Force  of  Bar  Pressing 

The  previous  chapter  argued  that  rats,  on  the  average, 
will  exert  relatively  more  force  to  obtain  relatively  sweeter 
reinforcements.  The  present  chapter  aims  to  extend  the  propor¬ 
tional  reinforcement  procedures  to  an  examination  of  the  role 
played  by  specified  gain  in  determining  level  of  attained. 

In  a  broad  sense  specified  gain  is  simply  the  ratio  of 
•payoff"  to' "exertion. "  Thus  gain  can  be  directly  controlled 
either  by  adjusting  the  exertion  requirement  or  by  changing  the 
amount  of  payoff  (or  both) .  Notterman  &  Mintz  (1965)  coined  the 
terra  "specified  gain"  in  order  to  describe  quantitative  differ¬ 
ences  among  the  proportional  reinforcement  schedules  used  in 
their  studies.  (In. this  discussion  gain  and  specified  gain 
will  be  used  interchangeably.)  However,  it  seems  clear  that  the 
notion  of  gain  is  implicit  in  any  experimental  work  which  uses 
cither  exertion  requirement  or  obtained  payoff  as  independent 
variables.  Generally  speaking,  most  of  the  previous,  research 
has  tended  to  emphasize  either  one  or  the  other  of  t|iese 
variables.  For  example,  Skinner  (1938)  demonstrated  that  increas¬ 
es  the  FR  requirement  would  eventually  bring  about  a  cessation 
responding,  regardless  of  whether  the  response  was  bar  press¬ 
es  or  wheel  running;  he  did  not  attempt  to  extend  the  upper 
limit  of  FR  responding  by  using  larger  or  more  preferred  rein- 
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forcemeats.  Another  more  recent  example,  is  Chung's  research 
(Chung,  1965)..  He  varied  the  force  requirement  for  microswitch 
closure  by  pigeons  and  found  that  rate  of  response  declined  with 
the  higher  force  requirements.  However,  Chung  did  not  observe 
the  effects  of  quantitative  variations  of  reward  on  this  function 
al  relationship.  Similarly,  the  many  studies  (see  Chapter  1, 
Table  1)  concerned  with  .amount  of  reinforcement  usually  have 
not  attempted  to  change  the  exertion  requirements,  although  Logan 
(1960)  has  studied  the  effects  of  pitting  delay  of  reward 
against  amount  of  reward. 

The  exploratory  work  reported  by  Notterman  &  Mintz  (1965, 
Chapter  11)  was  an  attempt  to  encompass  both  aspects  of  the  gain 
variable.  These  researchers  used  a  single  group  of  rats  and 
exposed  them  to  successive  force  proportional  schedules,  in 
which  number  of  pellets  obtained  varied  with  exertion.  However, 
the  minimum  force  required  for  some  reinforcement  was  always 
8.0  gm.  The  general  result  of  the  study  was  that  increasing 
the  gain  served  only  to  decrease  the  level  of  F  emitted.  This 

Occurred  probably  because  the  increase  in  gain  made  available 

» 

fcuch  more  reinforcement  per  response  (for  a  given  response 
distribution  this  would  also  increase  the  amount  obtained  per 
tospoasc)  ,  and  because  the  minimal  force  requirement  was  quite 
low.  Moreover,  in  terms  of  increased  forces  for  greater  amounts 
reinforcement,  the  rats  showed  no  responsiveness  to  the 
Proportional  reinforcement.  These  investigators  concluded  that 
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the  values  of  specified  gain  (i.e.,  an  index  based  on  the  mean 
of  the  individual  ratios  of  amount  (mg.)  of  reinforcement  to 
force  requirement  at  each  of  the  steps  of  a  particular  propor¬ 
tional  contingency)  were  so  high  that  there  was  no  particular 
advantage  to  the  animal  in  responding  differentially  to  the 
gpecified  gain  values  used. 

In  view  of  this  previous  work,  it  seems  clear  that  if  the 
gain  variable  actually  is  important  in  controlling  response 
force,  then  the  dynamic  range  in  which  it  operates  is  bounded  by 
the  extremes  of  zero  reinforcement  on  the  one  hand  and  an 
impossible  exertion  requirement  on  the  other.  Further,  if 
specified  gain  does,  at  some  point,  influence  force  of  response, 
it  may  well  lie  below  the  values  used  by  Hotter man  &  Mintz  (1965) . 
Thus,  choice  of  the  appropriate  quantitative  conditions  may  be 
crucial  if  the  supposed  effects  of  gain  are  to  be  observed. 

Thu  search  for  suitable  parameters  is  hindered  by  a  lack  of 
precise  psychophysical  scales  for  intensive  measures  of  respond¬ 
ing.  The  particular  values  of  concentration  of  sucrose  and 
force  requirements  used  in  the  present  work  represent  educated 
guesses,  the  “education"  deriving  primarily  from  the  previous 
chapter.  The  gain  values  for  Group  2,  Group  5,  and  Group  6 
*hown  in  Table  6,  seem  to  provide  sensible  jumping-off  places 
*0r  a  quantitative  assessment  of  gain.  A  2-step,  discontinuous 

Iroportional  reinforcement  schedule  was  chosen  because  it  yielded 

*  _ 

•  L  greatest  evidence  for  the  upward  shaping  of  response  forces 
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in  the  previous  chapter  (see  Fig.  7,  Chapter  3). 

Because  of  the  almost  infinite  number  of  possible  quanti¬ 
tative  arrangements,  and  because  of  the  limited  recording 
capability  of  the  apparatus,  the  restricted  extent  of  the  present 
study  of  gain  places  it  in  the  category  of  exploratory  or 
pilot  research.  For  this  reason,  the  several  combinations  of  force 
and  concentration  of  sucrose  were  tried  for  the  same  groups  of 
rats,  and  thus  the  experiment  does  not  fit  neatly  into  conven¬ 
tional  designs.  As  a  consequence,  the  description  of  the  soecific 
procedures  and  the  accompanying  results  are  discussed  chrono¬ 
logically  with  some  notice  being  given  to  the  performance  of 
individual  animals  at  each  phase.  Table  6  summarizes  the  various 
phases  and  the  associated  conditions.  The  specific  empirical 
question  to  which  this  chapter  addresses  itself  is:  Does  the 

specified  concentration  gain  influence  the  extent  of  high  force 
^nission  under  2-step  proportional  reinforcement  conditions? 

Method 

Apparatus 

The  response  recording  and  reinforcement  dispensing 

% 

apparatus  was  identical  in  all  respects  to  the  equipment  des¬ 
cribed  in  the  previous  chapter. 
fH-^jjucts  ana  Procedure  for  Phase  I 

The  subjects  were  20  rats  comparable  in  weight,  sox, 

^rain,  supplier,  and  deprivation  conditions  the  subjects 
Ascribed  in  the  previous  chapter. 
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phase  I,  which  consisted  of  shaping  of  the  bar-press  re¬ 
sponse,  was  carried  out  as  described  in  the  preceding  experi- 

•nunt.  The  rationale  for  using  this  procedure  was  given  pre- 

« 

viously*  Subsequent  to  Phase  I  the  subjects  were  randomly 
assigned  to  two  groups  of  10  each.  One  animal  of  Group  5 

alcd  toward  the  end  of  the  experiment,  so  none  of  its  data  are 

included  in  the  analysis. 

Results  and  Discussion  on  a  Phase-by-Phase  Basis 
fhase  II 

From  Table  6  it  can  be  seen  that  the  force  requirements 

for  Group  5  and  Group  6  are  identical  to  those  for  Group  2 

of  Chapter  3.  Each  of  these  three  groups  differ,  however,  with 

respect  to  the  gain  of  the  higher  force  requirement  and, 

therefore,  with  respect  to  the  gain  of  the  schedule  as  a  whole. 

If  the  gain  of  the  schedule  is  important  in  determining  the 

extent  to  which  rats  tend  to  press  harder  to  obtain  sweeter 

rewards,  then  one  would  expect  these  three  groups  to  display 

*igni£icant  intergroup  differences  in  F^  and  rank  themselves  in 

increasing  mean  F  as  follows:  Gp.  6,  Gp.  2,  and  Gp.  5. 

P 

Conversely,  lack  of  reliable  differences  would  imply  that  the 
'J*in  is  really  not  very  important  in  the  present  context,  or 
the  parametric  conditions  were  poorly  selected. 

Phase  II  consisted  of  15  consecutive  daily  sessions  so 

! 

‘^at  Group  5  and  Group  6  would  be  closely  comparable  to  Group 
*  in  terms  of  amount  of  training. 
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plotted  in  Fig.  11  are  the  mean  F  curves  for -Group  5 

? 

and  Group  6.  In  ‘Table  7  are  the  individual  subject  means  and 

standard  deviations  of  F  for  the  initial  and  final  sessions 

P 

of  Phase  II.  Both  groups  display  levels  of  mean  F  .  similar  to 

P 

those  of  Group  2,  suggesting  that  Group  5  and  Group  6  were 
affected  in  an  upward  direction  by  exposure  to  the  proportional 
reinforcement,  as  was  Group  2.  (That  is,  the  forces  remained 
higher  than  the  control  groups  throughout  Phase  II.)  However, 
a  one-way  analysis  of  variance  conducted  on  threse  three  groups 
for  the  last  three  sessions  combined  of  Phase  II  yields  a  non¬ 
significant  F  (2,  24)  -  1.0315.  The  implication  is  that  the 
differences  in  gain  were  insufficient  to  produce  reliable 
differences  among  the  three  groups  in  terms  of  mean  F^  of 
response.  Nor  were  there  any  significant  differences  among 
these  three  groups  in  rate  of  responding  (see  Fig.  12) .  Here 
also  a  one-way  analysis  of  variance  resulted  in  an  unreliable 
P  (2,  24)  =  0.5104.  However,  the  forces  were  distributed  in  a 
banner  that  did  result  in  cignfiicant  intergroup  differences 
in  rate  of  reinforcement,  F  (2,  24)  =  7.2651,  p  <  .01.  This 
iatter  finding  is  a  natural  consequence  of  the  similar  behavior 
of  the  three  groups  with  regard  to  force  and  rate.  All  three 
groups  were  taking  advantage  of  the  proportional  reinforcement 
*°  approximately  the  same  degree,  and  the  differences  in  rein- 

i 

forcement  rate  simply  arose  from  the  different  amounts  available 
undGr  each  reinforcement  contingency. 


Figure  11.  Mean  as  a  function  of  the  four  experimental 
phases.  Each  point  is  the  group  mean  of  the  individual  subject 
moans  for  a  single  session.  Compare  the  "curves"  of  Phase  II 
vith  those  plotted  in  Fig.  7,  Chapter  3. 
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Mean  and  Standard  Deviation  of  P  and  Number  of  Responses 
for  Initial  and  Final  Seslions  of  Phase  II 


Initial  Final 


r.rnup  Subject 

Mean 

S.D. 

N 

Mean 

S.D. 

N 

t 

-  5 

22.0 

16.0 

193 

19.3 

14.3 

198 

-1.728 

11 

14.6 

11.0 

229 

32.5 

16.8 

166 

11.978**** 

1  13 

17.7 

9.4 

186 

16.3 

7.4 

181 

-1.569 

21  14 

15.8 

13.6 

236 

28.2 

32.2 

233 

5.406*** 

15 

20.1 

13.6 

205 

24.0 

20.8 

225 

2.311* 

23 

26.1 

22.3 

189 

30.3 

23.6 

155 

1.706 

30 

23.1 

19.9 

201 

27.7 

33.5 

237 

1.795 

33 

Mean 

11.1 

18.8 

8.6 

268 

11.1 

23.7 

4.2 

187 

-0.098 

42 

16.5 

15.3 

236 

22.8 

19.4 

229 

1.816 

43 

15.  8 

13.7 

248 

27.9 

15.8 

186 

8.405**** 

44 

15.8 

12.4 

200 

16.2 

9.0 

197 

0.368 

47 

18.4 

11.1 

188 

24.4 

17.4 

203 

4.094*** 

5  48 

14.4 

7.7 

180 

20.4 

10.8 

167 

5.889*** 

57 

25.5 

20.4 

185 

14.8 

10.8 

232 

-6.448**** 

59 

* 

22.3 

17.1 

197 

15.9 

10.6 

190 

-4.458*** 

60 

23.0 

18.5 

220 

19.2 

10.0 

186 

-2.670** 

61 

Mean 

21.4 

19.2 

17.0 

214 

31.8 

21.5 

21.7 

178 

5.237*** 

40 

37.6 

18.5 

157 

22.3 

12.1 

166 

-8.769**** 

49 

19.4 

12.7 

178 

21.2 

13,1 

166 

1.257 

50 

15.3 

11.0 

201 

18.4 

7.9 

163 

3.126** 

52 

17.4 

13.2 

215 

13.4 

8.3 

228 

-3.769** 

6  54 

21.9 

18.1 

220 

29.2 

20.7 

192 

3.790** 

55 

14.6 

12.6 

266 

17.4 

9.1 

175 

2.696* 

56 

32.6 

16.1 

152 

12.5 

7.2 

199 

-14.265**** 

58 

15.7 

12.6 

246 

18.2 

12.4 

219 

2.101* 

62 

22.3 

16.6 

201 

19.6 

9.4 

169 

-1.946 

63 

Mean 

22.5 

21.9 

12.8 

188 

23.5 

19.6 

10.3 

164 

0.794 

t 


^tes: 

1.  These  data  are  repeated  from  Table  4. 

2.  means  decrease,  *p  <  .05,  **p  <  .01,  ***p  <  .0002, 

****p  <  .000001. 
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f 

I 


Figure  12.  Average  rate  of  responding  as  a  function  of  the 
four  experimental  phases.  Each  point  is  a  group  mean  based 
on  the  response  rate  of  separate  subjects  during  one  session. 
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Examination  of  the  individual  performances  suggests, 

as  in  Chapter  3,  that  some  subjects  simply  do  not  find  it 

advantageous  to  respond  at  the  higher  force  levels  despite  the 

sweeter  rewards  which  these  responses  can  provide.  For 

instance,  number  56  of  Group  6  displays  a  mean  F  similar  to 

P 

that  expected  for  CRF  with  an  8.0  gm.  criterion-.  On  the  other 
hand,  some  subjects,  for  example,  number  54  of  Group  6,  clearly 
seem  to  be  regularly  emitting  the  high  forces  required  for  the 
sweeter  solution.  These  large  individual  differences  again 
call  attention  to  the  possibility  that  gain  may  lack  sufficient 
potency  to  be  considered  as  an  important  indpendent  variable 
in  controlling  response  force. 

Phase  III 

In  this  phase  the  gain  for  both  Groups  5  and  6  was  reduced, 
as  shown  in  Table  6,  by  increasing  the  higher  force  requirement. 
Although,  this  tactic  decreased  the  gain,  it  did  not  influence 
the  relative  difference  between  the  two  groups.  One  advantage 
of  manipulating  the  gain  by  increasing  the  force  requirement 
Is  that  it  should  produce  large  differences  in  mean  between 
subjects  which  continue  to  press  harder  to  obtain  the  sweeter 
solution  and  those  which  do  not. 

Group  data  for  mean  F^  of  response  for  the  15  daily 
sessions  of  this  phase  are  presentee  in  Fig.  11.  Despite  the 
ouvious  graphic  separation,  the  intergroup  comparison  of  mean 
is  not  nearly  significant  (last  session  of  Phase  IIx, 
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p  >  .10,  Wilcoxon's  Rank  Sum  Test;  session  9,  where  graphic 
'  2 

differences  are  greatest,  p  >  .10)  .  Similarly,  rate  of  response 

showed  no  reliable  intergroup  differences  (p  >  .10,  see  Fig.  12). 

In  view  of  the  individual  differences  displayed  previously, 

it  seems  plausible  that  the  group  curves  may  actually  represent 

the  lumping  together  of  two  more  or  less  separate  populations 

within  each  of  the  groups  -  one  of  these  populations  pressing 

to  get  the  sweeter  solutions,  the  other  not.  This  possibility 

suggested  that  a  plot  of  the  individual  "learning  curves" 

for  mean  F  for  Phase  III  might  be  illuminating.  Accordingly, 

P 

these  data  are  shown  in  Fig.  13.  If  the  higher  gain  schedule 
is  having  an  effect  on  mean  F^  and  if  responsiveness  to  propor¬ 
tional  reinforcement  is  displayed  in  an  all-or-none  fashion, 
then  one  would  expect  the  influence  of  gain  to  make  itself 
manifest  in  terms  of  a  relatively  greater  number  of  subjects 
showing  high  forces  v/ith  comparatively  high  gain.  This 
hypothesis  appears  to  be  borne  out  by  the  data,  but  not  dramati¬ 
cally.  Three  subjects  in  Group  5  (numbers  42,  43,  and  61) 


f!on-parametric  statistics  are  used  in  this  and  the  subsequent 
phases  because  of  the  large  differences  in  variance  between  the 
*wo  groups  in  mean  F  .  By  the  Seigel-Tukey  non-parametric 
variability  test  (WiScoxon  &  Wilcox,  1964),  Group  5  is  more 
variable  than  Group  6,  p  <  .02.  This  points  up  the  inappropriate- 
n°Ss  of  a  t  tost.  Even  with  non-parametric  procedures  the 
Problem  of  multiple  applies* ion  of  tests  to  the  same  data  can- 
^°t  be  avoided.  Thus  the  intergroup  comparisons  are  to  be 
^terpreted  cautiously,  for  each  successive  test  weakens  the 
icvel  of  confidence  in  an  unspecified  way  (see  Hays,  1963). 


Figure  13.  Mean  F^  for  individual  subjects  as  a  function 
of  Phase  III  conditioning.  Each  curve  is  labeled  with  the  subject 


number.  These  curves  may  be  compared  with  the  individual  data 
given  in  Table  7  for  Phase  II. 


-^iON^ 


and  one  subject  in  Group  6  (number  54)  probably  were  attempting 
to  meet  the  higher  force  requirement.  However,  even  if  these 
differences  in  the  number  of  subjects  displaying  high  mean  F 
do  represent  the  effects  of  the  independent  variable  gain, 
such  small  intergroup  differences  imply  that  gain,  as  opera¬ 
tionally  defined  here,  is  not  a  particularly  influential 
independent  variable.  'll  the  other  subjects  were  essentially 
responding  as  if  the  8.0  gm.  requirement  was  the  only  contin¬ 
gency  operative.  In  fact,  these  latter  subjects  displayed 
substantial  decreases  in  mean  during  Phase  III.  By  pooling 
the  data  of  both  groups  and  by  comparing  the  three  sessions  of 
Phase  II  with  the  final  three  sessions  of  Phase  III,  a  rough 
overall  estimate  of  this  tendency  to  decrease  in  mean  F^  can  be 
made.  Wilcoxon's  signed  rank  test  applied  to  the  percent  change 

in  mean  F  does  give  indication  of  a  reliable  decrease 
P 

(p  <  .05). 

In  one  sense,  the  general  downward  trend  in  mean  F^  can 

ho  interpreted  as  gain  "optimizing"  behavior  simply  because 

the  highest  gain  value  represented  in  either  proportional 

Schedule  is  1.0  for  the  lower  force  step,  as  opposed  to  .80  and 
the  gain  values  of 

*40,  which  are/the  higher  force  steps  for  Group  5  and  Group  6 
respectively.  As  a  cautionary  note,  it  should  be  recalled  that 
numerical  values  for  gain  probably  do  not  fairly  represent 
the  "real" ' situation  as  experienced  bv  the  rat.  Thus  the  down- 
ward  trend  may  be  a  reflection  of  a  least  effort  effect.  How- 
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ever#  this  view  would  not  explain  why  three  subjects  in  Group 
5  and  one  subject  in  Group  6  pressed  at  the  higher  levels.  The 
forces  of  these  subjects  are  so  high  that  their  occurrence 
seems  to  be  beyond  that  plausibly  accounted  for  by  random 
variation  alone.  A  possible  resolution  to  this  dilemma  hinges 
on  a  supposition  that  the  upper  force  requirement  was  simply 
beyond  the  physical  capacity  of  the  majority  of  subjects. 

(During  the  shaping  phase,  however,  most  subjects  did  emit  a 
few  forces  of  such  magnitude.)  Perhaps,  they  gradually  came 
to  respond  "efficiently"  to  the  only  contingency  actually 
available  to -them,  i.e.,  the  8.0  gm.  criterion.  The  validity 
of  this  alternative  conjecture  is  examined  in  Phase  IV. 

Phases  IV  and  V 

In  these  phases  both  groups  were  switched  to  the  regular 
reinforcement  schedule  (as  opposed  to  proportional  reinforce¬ 
ment)  as  shown  in  Table  6.  The  only  responses  which  received 
reinforcement  (.05  cc,  of  8%  sucrose)  wore  those  equal  to  or 

above  40  gm.  Use  of  this  schedulo  in  the  present  context 

\ 

should  help  determine  whether  the  performances  of  the  previous 
phase  were  a  result  of  a  physical  incapacity  to  press  with 
responses:  of  at  least  40  gm.  ,  or  whether  the  general  decline 
^  forces  was  dictated  by  the  relative  attractiveness  of  the  gain 
the  lower  requirement  (which  in  this  case  is  indistinguishable 
from  a  tendency  for  the  animals  to  exert  "least  effort" 

Compatible  with  at  least  some  reinforcement.) 


Three  subjects  (one  in  Group  5  and  two  in  Group  6)  showed 


signs  of  extinction  during  the  course  of  Phase  IV,  But  the 
remaining  subjects  increased  their  forces  to  meet  the  new 
requirement  a  largo  proportion  of  the  time  (Wilcoxon's  Signed 


Rank  Test,  based  on  mean  F  of  subjects  responding  at  the  end 

P 


of  Phase  IV,  p  <  .05  and  p  <  .02  for  Group  5  and  Group  6, 
respectively} .  Therefore,  the  40  gm.  requrement  probably  does 
not  pose  an  impossible  task  for  the  majority  of  rats;  however, 
it  may  be  very  close  to  the  exertion  level  which  will  just 
barely  maintain  responding. 

Accompanying  the  snift  to  Phase  XV  was  an  increase  in  rate 
of  responding  though  not  quite  a  reliable  one  (Wilcoxon’s 
Signed  Rank  Tost  for  both  g r  combined,  p  <  .10).  However, 

as  training  on  the  40  gm.  criteron  ensues  and  as  forces  increase 
there  is  a  very  large  decline  in  rate  of  responding  (Wilcoxon's 
Signed  Rank  Test  for  both  groups  combined,  p  <  .01).  At  first 
glance,  these  data  seem  to  be  compatible  with  the  results  for 
pigeons  reported  by  Chung  (1965).  Indeed,  if  there  is  a 


genuine  tendency  for  mean  P  and  rate  of  response  to  be  negative 

P 


ly  related,  then  one  would  expect  such  an  effect  to  manifest 
itself,  on  a  within  group  basis,  in  the  form  of  a  significant 
negative  correlation  coef f icieiit.  Accordingly,  Spearman's 
r^nk  correlation  coefficient  was  computed  for  the  fourth  session 
Phase  IV,  the  result,  r  =  -0.6205,  being  significant  at  the 
level  of  confidence.  (The  three  animals  which  essentially 


12 


extinguished  were  not  included  in  this  computation.)  Two  factors 
contribute  to  this  negative  relationship  between  rate  and  mean 

F  .  First,  the  number  of  sub-criterion  responses  (those  between 

P 

2.5  and  39.3  gra.)  decreased  as  the  level  of  Fp  increased  during 
Phase  XV.  Second,  the  high  forces  appear  to  bring  about  a 
general  slow  down  in  responding  perhaps  stemming  from  fatigue. 

It  is  possible  that  the  individual  cases  which  extinguished 
during  Phase  IV  did  so  simply  because  of  the  drive  operations 
used.  To  check  on  this  eventuality  the  subjects  were  not  fed 
following  the  fourth  session  of  Phase  IV.  The  effects  of  the 
drive  shift  were  observed  on  the  following  day  which  was' the 
single  session  devoted  to  Phase  V.  Rate  of  responding  increased 
for  the  individuals  which  had  almost  ceased  to  respond  during 
Phase  IV;  however,  their  forces  showed  further  decreases  sug¬ 
gesting  that  these  subjects  could  not  emit  the  requisite  forces, 
or  if  they  could,  8.0%  sucrose  was  an  insufficient  inducement  to 
so.  Excluding  these  three  subjects,  overall  level  of  F 

P 

remained  essentially  unchanged  (Wilcoxon's  Signed  Rank  Test, 

P  >  .10),  and  rate  of  response  also  did  not  change  significantly 
(same  test,  p  >  .10),  although  it  did  tend  to  increase.  Such 

*  large  shift  in  the  drive  conditions  failed  to  influence 

* 

Pliably  cither  rate  or  mean  F  probably  because  the  rats  were 

ir 

already  very  near  maximum  output  by  the  fourth  session  of 
*^ase  iv. 

The  data  collected  during  Phases  IV  and  V  are  particularly 
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noteworthy  in  two  respects.  In  the  first  place  it  seems  remark¬ 
able  that  .05  ml.  of  8,0%  sucrose  can  maintain  responding  on 
such  a  high  force  criterion;  i.e.,  the  ratio  of  payoff  to 
exertion  was  extremely  low.  Second,  the  drive  operations 
of  22  hours  of  food  deprivation  appears  to  have  brought  about 
near  maximum  behavioral  output  in  terms  of  rate  and  mean  F 

r 

of  response. 

Taken  together,  the  data  from  this  experiment  indicate 
the  following:  The  results  obtained  for  Phase  II  suggest  that 
most  rats  found  it  "worthwhile"  to  emit  quite  often  forges  above 
24.0  gm. ,  regardless  of  whether  16.0%,  24.0%.  or  32.0%  rein¬ 
forced  these  responses.  Since  forces  between  8.0-23.9  gm. 
yielded  uniformly  only  8.0%  concentration,  the  finding  emphasizes 
the  behavioral  control  taken  by  concentration  rather  than  by 
gain.  However,  the  presence  of  the  overall  downward  trends  for 
mean  in  Phase  III  suggests  that  the  average  subject  may 
consistently  choose  the  higher  of  two  gain  values  when  it  is 
paired  with  low  exertion.  Such  seems  to  be  the  case,  even 
though  when  required  to  do  so,  subjects  will  regularly  emit 
*0  gm.  or  more  to  obtain  only  8.0%  solutions. 
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Chapter  5 

Feedback  during  Regularly  and 
Proportionally  Reinforced  Responding 

Experimental  psychology  has  as  one  of  its  tasks  the 
elucidation  of  laws  controlling  behavior.  The  word  "control" 
immediately  suggests  that  certain  aspects  of  experimental 
psychology  can  be  regarded  from  a  cybernetic  point-of-view. 

(As  defined  by  Norbert  Wiener,  (1948),  cybernetics  means 
communication  and  control  in  the  animal  and  the  machine.  Ad¬ 
mittedly,  this  is  an  extremely  broad  definition.  In  current 
usage,  however,  cybernetics  connotes  a  point  of  view  more  than 
it  describes  a  particular  area  of  study) .  Vital  to  a  cybernetic 
approach  to  behavior  analysis  are  feedback  models.  These 
extend  in  degree  of  formality  from  mathematical  models  borrowed 
from  mechanical  and  electrical  control  theory  (McFarland,  1965), 
to  less  structured,  verbal  and  diagramatic  schemes  (Notterman, 
1970).  Such,  models  have  proved  their  heuristic  value  in 
modern  psychology.  For  example,  Smith  (1962)  has  shown  that 
sensory  feedback  is  essential  to  normal  perceptual-%otor  function 
log.  in  the  field  of  operant  behavior,  Notterman  &  Mintz  (1965) 
have  demonstrated  the  utility  of  viewing  certain  aspects  of  bar 
Pressing  as  being  dependent  on  feedback  processes.  It  is  the 
purpose  of  the  present  chapter  to  attempt  to  develop  quanti¬ 
tative  descriptions  of  some  of  the  feedback  processes  suggested 
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by  Notterman  (1970) ,  as  well  as  by  the  present  author. 

In  Fig.  14  is  a  diagram  which  depicts  three,  more  or  less 
distinct,  feedback  loops  which  may  be  important  during  a  session 
of  bar  pressing  for  food  on  an  isometric  silent  manipulandum. 

One  of  these  loops,  which  is  represetned  by  the  connections 
between  the  blocks  labeled  A,  B,  and  C,  is  primarily  intra- 
organismic.  That  is,  the  loop  is  physically  continuous  and  is 

t 

constructed  of  nerves  and  muscles.  Quite  simply,  this  feedback 
allows  the  organism,  while  responding,  to  make  use  of  sensory 
information  which  is  generated  by  the  response  itself.  The 
data  collected  in  the  previous  chapters  and  analyzed  in  the 
present  one  are  relevant  to  this  particular  loop,  but  do  not 
afford  a  precise  description.  Thejtwo  loops  which  include  extra 
organismic  components  are  the  primary  concern  of  the  present 
chapter.  The  first  of  these  extra-organismic  loops  can  be 
referred  to  as  a  satiation-hunger  loop.  It  is,  of  course, 
represented  by  the  food  reinforcement  feeding  back  to  reduce 
hunger  thereby  decreasing  the  tendency  to  emit  responses 
(connections  among  blocks  D,  E,  C,  F,  and  H) .  A  second  extra- 
organismic  loop,  also  involving  the  consequences  of  the  response 
is  derived  from  the  idea  that  reinforcement  is  a  stimulus  as 
Veil  as  a  drive  reducer,  (Henceforth,  to  be  called  reinforce- 
fcont-response  loop  (connections  among  E,  C,  and  F] . )  That  is, 
the  events  following  a  response  may  influence  a  subsequent 
tcsponse  independently  of  drive  level.  The  block  diagram  in 


Figure  14.  Block  diagram  depicting  three  possible  feed¬ 
back  loops  which  may  be  important  avenues  of  information  during 
a  session  of  bar  pressing  on  a  silent,  isometric  manipulandum. 
Connections  among  blocks  h,  B,  and  C  represent  an  intra- 
organismic  loop,  allowing  the  response-induced  feedback  to  be 
sensed  while  the  response  is  being  emitted.  Connections  among 
blocks  H,  D,  E,  C,  and  F  represent  the  satiation-hunger  loop. 
The  lines  joining  blocks  E,  C,  and  F  portray  the  reinforcement- 
response  loop. 
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pig,  14  is  not  to  be  taken  too  literally;  it  is  presented  for 
the  sake  of  exposition.  Undoubtedly  the  "real"  situation  is 
fantastically  more  complex  than  the  diagram  suggests. 

Since  this  chapter  is  to  examine  the  loops  containing 
extra-organismic  components,  it  is  important  to  inquire  why 
two  such  loops  are  proposed  as  being  necessary.  The  reason  for 
considering  the  satiation  -  hunger  loop  is  straightforward:  In 
many  experimental  studies  using  hungry  animals  it  has  been  ob¬ 
served  that  rate  of  responding  decreases  as  the  animals  procure 
more  and  more  food.  Thus  it  is  natural  to  ask. whether  the 
effect  of  drive  reduction  will  manifest  itself  also  in  terms  of 
another  dependent  variable,  peak  force  of  response. 

The  rationale  behind  the  second  loop  with  extra-organismic 
components  is  not  quite  as  simple.'  Often,  the  time  rate  at 
which  food  reinforcement  is  obtained  is  high  enough  to  allow 
the  procurement  of  several  reinforcements  before  any  significant, 
direct  drive  reducing  effects  can  occur.  It  is  quite  possible 
that  the  stimulus  properties  of  food — in  that  they  influence 
taste — can  bring  about  physiological  changes,  e.g.,  release  of 
insulin  from  the  pancreas,  which  may  influence  thk  organism's 
nutritive  needs  {Valenstein  &  Weber,  1965) .  Thus  it  appears 
nsasonable  to  posit  (conceptually)  two  separate  loops  by  which 
the  effects  of  reinforcement  influence  the  rate  and  intensity 
of  subsequent  responses. 

The  importance  of  the  stimulus  properties  of  reinforcement 
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in  determining  mean  peak  force  of  response  was  demonstrated  in 
Chapter  3.  Proportional  reinforcement  (specified  in  terms  of 
concentration  of  sucrose)  produced  higher  mean  peak  forces  than 
amount  -  proportional  reinforcement.  However/  the  extent  to 
which  a  single  reinforcement  influences  the  very  next  responses 
has  yet  to  be  examined.  The  typical  subject  in  Group  II 
(8.0  gm.  to  23.9  gm.  8.0%,  and  24.0  +  gm.  +  24.0%),  because 
of  variability  in  response  force,  obtained  sequences  of  rein¬ 
forcements  in  which  different  concentrations  frequently  followed 
one  another.  Did  these  discriminably  different  reinforcements 
produce  "extinction  effects"  or  "amount  of  reinforcement- effects"? 
TS  --it  is,  from  several  experiments  (Notterman  &  Mintz,  1965)  it  is 
n noi.-n  that  reduction  of  reinforcement  to  zero  amount  brings 
about  a  rapid  rise  in  mean  peak  force.  Could  such  a  phenomenon 
on  a  smaller  scale  be  occurring  during  proportionally  rein¬ 
forced  responding?  Does  the  receipt  of  an  8.0%  solution  compare 
bo  unfavorably  with  a  24,0%  solution  that  the  rat  presses 
harder  after  obtaining  an  8.0%  reinforcement  and  vice  versa? 

Do  any  regular,  distinguishable  patterns  develop  as  a  result  of 
such  tendencies,  leading  to  a  cyclical  form  of  response 
stereotypy?  And  are  the  supposed  patterns  in  early  training 
different  from  those  in  later  training?  Are  different  patterns 
correlated  with  the  various  reinforcement  schedules?  Answers  to 
these  questions  are  sought  by  examining  the  serial  properties 
°f  response  force  emission  through  the  use  of  spectral  analysis. 


The  Satiation-Hunger  Loop 

Because  of  the  time  required  for  food  to  be  assimilated  and 
because  reinforcements  are  accumulated  over  session  time,  it 
is  reasonable  to  assume  that  evidence  for  the  presence  of 
satiation  might  manifest  itself  in  terms  of  a  trend  in  the  peak 
forces  emitted  during  a  session.  Thus  linear  regression 
analysis  was  applied  to  the  session  peak  force  records  of 
Phase  II  for  all  subjects  of  the  six  experimental  groups 
reported  in  Chapters  3  and  4. 

Before  the  analysis  was  undertaken,  it  was  necessary  to 
consider  the  possibility  that  the  peak  force  distributions  may 
require  a  normalizing  transformation  in  order  to  conform  to 
the  assumptions  underlying  parametric  statistical  tests.  In 
Fig.  15,  mean  peak  force  is  plotted  as  a  function  of  standard 
deviation  of  peak  force  for  the  individual  subjects  of  all  six 
groups.  It  is  clear  that  there  is  a  strong  relationship 
between  mean  and  standard  deviation.  (Correlation  coefficients 
computed  for  each  group  separately  are  of  comparable  magnitudes.) 
Given  this  robust  association  and  the  fact  that  force  (when 
considered  as  a  scalar)  is  an  essentially  positive  quantity 
(see  Jenkins  &  Watts,  1968),  a  log^Q  transformation  was  tried. 
Figure  16  shows  that  the  transformation  was  partially  successful 
in  reducing  the  correlation  between  the  mean  and  standard 
deviation.  In  view  of  the  moderate  success  of  taking  logarithms, 
no  other  transformation  was  attempted,  and  the  linear  regression 


Figure  15.  Scatter  plot  of  mean  F^  and  standard  deviation 

of  F  .  Each  point  is  based  on  the  total  distribution  o£<re- 
P 

spones  for  the  final  three  sessions  of  Phase  II.  Data  for  all 
the  subjects  of  Groups  1  through  6  are  plotted. 
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Figure  16.  Scatter  plot  of  the  mean  and  standard  devia¬ 
tion  of  the  log10  transformed  peak  force  distributions  for  the 
last  three  sessions  of  Phase  II.  Data  for  all  subjects  in  Groups 
1  through  (>  ara  presented. 
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analysis  was  conducted  on  the  log.^  transformed  scores. 

The  observed  relationship  between  mean  and  standard 
deviation  can  be  explained  in  a  number  of  ways:  From  a  psycho¬ 
physical  viewpoint  the  association  is  a  natural  consequence  of 
Weber’s  law.  Or  one  could  argue  that  low  magnitude  responses 
have  been  reinforced  in  the  past,  especially  during  the  shaping 
phase,  and  as  e.  result  are  emitted  occasionally,  thereby  inflat 
ing  the  variance  when  the  rat  is  responding  at  relatively  high 
force  levels.  This  latter  type  of  interpretation  is  similar  to 
Skinner's  (1938)  concept  of  the  reflex  reserve  and  Keller  & 
Schoenfeld's  (1950)  idea  of  response  subclasses. 

Regression  coefficients  of  F^  on  ordinal  position  are 
presented  in  Table  8.  It  is  immediately  obvious  that  the 
coefficients  are  quite  small  with  _most  of  the  slopes  being  very 
near  zero.  If  there  were  some  slight  reliable  trend,  one  would 
expect  the  mean  regression  coefficent  for  a  particular  group 
to  differ  significantly  from  zero.  Accordingly,  this  hypothesis 
was  tested  using  the  t  distribution.  Out  of  the  24  tests 
carried  out,  only  one  was  significant  at  the  .05  confidence 
level  which  is — of  course — what  would  be  expected  by  t  distri¬ 
bution  in  the  absence  of  any  true  difference.  Thus  there  is  no 
hard  evidence  that  the  data  display  any  reliable  linear  trends 

during  a  session,  either  early  in  conditioning  or  late  in 

* 

conditioning. 

The  plot  of  Fp  as  a  function  of  ordinal  position  for  a 
typical  subject  frorfl  Group  II  (Fig.  17)  provides  a  visual  demon- 


‘Table  8 


Regression  Coefficients  of  Iiog^^  on  Ordinal  Position 


croup 


Sessions  of  Phase'  2 


>jects 

1 

2 

14 

15 

3 

.069 

.032 

.029 

.003 

9 

.044  ' 

-.062 

-.007 

-,002 

16 

-.007 

.000 

-.018 

.018 

17 

.026 

-.034 

.002 

.  006 

21 

-.011 

.055 

.022 

.  -.011 

29 

-.015 

-.009 

.016 

-.040 

35 

-.036 

-.024 

.036 

.028 

Mean 

.020 

-.006 

.011 

.00 

5 

-.032 

-.005 

.008 

-  .024 

11 

-.002 

.005 

-.026 

-.014 

13 

-.047 

.005 

-.016 

.  015 

14 

-.031 

-.012 

.005 

.026 

15 

-.014 

.004 

.000 

.000 

23 

.015 

.067 

.000 

X 

30 

-.056 

-.014 

-.019 

-.002 

33- 

.024 

.013 

-.017 

.013 

Mean 

-.010 

.008 

-.008 

.009 

1 

-.002 

.072 

.025 

-.004 

2 

.029 

-.065 

.019 

.003 

6 

-.033 

-.026 

.018 

-.018 

8 

-.029 

-.023 

.019 

-.004 

20 

-.007 

.011 

-.010 

-.017 

28 

.030 

.003 

-.006 

-.015 

36 

-.027 

-.040 

.005 

-.053 

38 

-.006 

.005 

-.027 

-.032 

Mean 

-.005 

-.008 

.005 

-.018* 

4 

-.049 

-.003 

.033 

,003 

10 

.  .007 

-.037 

-.009 

-.038 

18 

.016 

.007 

.021 

.000 

25 

-.037 

-.019 

-.006. 

-.018 

26 

.019 

.031 

-.051 

-.024 

31 

.008 

.018 

.026 

.033 

34 

-.031 

.005 

-.019 

.027 

Mean 


010 


000 


002 


-.003 


Table  8  (Cent.) 


42 

.024 

.046 

.022 

.027 

43 

.007 

-.021 

-.021 

.001 

44 

.005 

.010 

-.031 

.009 

47 

-.010 

.025 

-.026 

.030 

48 

.010 

.000 

.019 

.012 

57 

-.059 

-.013 

-.022 

-,028 

59 

“,034 

-.048 

-.015 

-.0.11 

60 

.005 

.008 

.006 

.005 

61 

-.008 

-.036 

.037 

.037 

Mean 

“.007 

-.003 

-.003 

.009 

4C 

.001 

.019 

,010 

.037 

49 

-.003 

-.035 

-.003 

.015 

50 

~,008 

.021 

.033 

.053 

52 

-.002 

.001 

.018 

-.020 

54 

.('57 

-.044 

.000 

.001 

55 

-.021 

.010 

.037 

.001 

56 

.  .032 

.026 

-.006 

.024 

58 

“.015 

-.005 

-.002 

.000 

62* 

-.009 

.068 

-.036 

-.054 

63 

.028 

.053 

.109 

.043 

Mean 

.006 

.011 

.016 

.010 

Notes : 

1.  *  p  <  .05,  null  hypothesis  of  zero  sJope  using  to  the  t 

distribution. 

"x"  data  lost  in  card  reads...  jam. 
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Figure  17.  Peak  force  of  response  for  one  session  of 
responding  in  Phase  II.  This  subject  was  in  Group  2.  *. 
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stration  of  the  failure  of  linear  regression  to  account  for  an 
appreciable  portion  of  the  variance.  The  most  prominent  feature 
of  this  graph  is  the  large  amount  of  variability.  Are  the  large 
changes  in  peak  force  correlated  with  one  another  and  with  the 
type  of  reinforcement  which  these  responses  produce?  Do  serial 
correlation  and  spectral  analysis  account  for  any  more  of  the 
variance  than  linear  regression?  Or  are  the  observations 
essentially  uncorrelated?  These  questions  are  examined  in  the 
next  section. 

The  Reinforcement-Response  Loop  and  Spectral  Analysis 

Since  Abelson’s  (1953)  pioneering  efforts  to  apply  spectral 
analysis  to  psychological  phenomena,  groat  strides  have  been 
made  in  the  general  development  of  these  analytic  techniques. 

The  physical  and  information  sciences  have  benefited  from  these 
improvements  more  than  has  experimental  psychology.  However,  in 
recent  years  there  have  been  a  few  attempts  to  use  spectral 
analysis  on  data  gathered  from  experiments  emphasizing  psycho- 
ogica)  variables  (e.g.,  Weiss,  Laties,  Siegel,  6  Goldstein, 

’^66).  Subsequent  to  Abelson's  dissertation  several  excellent 
-ooks  have  been  written  which  make  spectral  analysis  accessible 
to  the  non-mathematician  (see,  e.g.,  Blackman  &  Tukey,  1958} 
feondat  &  Piorsol,  1966}  Fishman,  1968;  and  Jfonkins  &  Watts, 

*568) .  m  view  of  the  extensive  mathematical  treatment  this 
topic  has  received,  the  author  will  make  no  attempt  to  give  a 
totalled  explication  of  Fourier  analysis  and  its  statistical 
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counterpart,  spectral  analysis.  Rather,  for  the  sake  of 
brevity  and  ease  of  communication,  the  explanation  is  developed 
on  a  verbal  and  graphic  basis. 

Spectral  analysis  is  a  type  of  time  series  analysis  which 
attempts  to  discover  anddescribe  the  periodic  tendencies  of  a 
time-based  process.  In  the  experimental  sciences,  one  frequently 
encounters  data  which  consist  of  measurements  ordered  in  time; 
these  observations  as  a  function  of  time  are  known  as  a  "time 
series. *  In  some  cases  the  data  are  sufficiently  consistent  so 
that  a  mathematical  function  can  be  used  as  a  model  for  the 
process.  An  example  of  such  a  process  can  be  represented  by 
a  cosine  function: 

y{t)  a  a  cos  {2  it  fQt) 

where  y(t)  is  the  process,  f  is  the  frequency  of  oscillation 
in  cycles  per  unit  time,  and  a  is  the  maximum  amplitude  of 
the  process.  A  well-regulated  60  cps  (cyclos/sec  or  hertz) 
alternating  current  voltage  can  be  approximated  by  such  a 
function.  The  rhythmic  variations  in  voltage  can  also  be 
described  in  terms  of  the  period  of  oscillation,  which  is  the 
time  interval  required  for  the  process  to  complete  one  cycle. 
ror  a  60  cps  alternating  voltage  the  period,  or  the  reciprocal 
of  the  frequency,  is  1/60  cps  or  about  16.7  milliseconds. 
Unfortunately,  the  time  series  encountered  in  a  practical  research 
6c'tting  are  not  nearly  so  well-behaved  as  the  cosine  function. 
Indeed,  sometimes,  especially  in  experimental  psychology,  it  is 
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not  at  all  obvious  that  an  observed  series  has  any  orderly 
aspects.  Measurement  error  and  uncontrolled  factors  combine 
to  introduce  random  components  into  the  observations  of  any 
realizable  time  series.  In  many  cases,  phenomena  occurring  in 
time  include,  to  greater  or  lesser  extents,  random  fluctuations 
(e.g.,  a  Guassian  process)  that  can  be  described  by  their 
statistical  properties.  .These  are  usually  called  stochastic 
processes  and  are  characterized  by  the  probability  distributions 
associated  with  them,  known  statistically  as  random  variables, 
X(t).  Thus  the  stochastic  process  is  described  by  a  set  of 
random  variables  such  that  for  each  point  in  time  there  is  a 
probability  distribution  with  a  given  mean,  variance,  and 
higher .moments.  An  example  of  a  well-described  stochastic 
process  would  be  the  cosine  function  plus  a  Gaussian  component: 

X(t)  =  a  cos  (2  it  fQt)  +  Z(t) 

where  X(t)  is  the  random  variable  representing  the  stochastic 
process,  and  Z(t)  is  a  Gaussian  process.  The  Gaussian  process 
has  a  normal  probability  distribution,  and  successive  observa¬ 
tions  in  time  are  essentially  uncorrelated.  It  is  called  white 
noise  by  communicatiois  engineers. 

.  Prom  the  point  of  view  of  experimental  science  the  usual 
problem  is  to  detect  the  orderly  (e.g.,  periodic,  rhythmic, 
patterning)  aspects  of  a  stochastic  process  as  well  as  to  des¬ 
cribe  the  probability  distributions.  On  an  intuitive  level,  it 
seems  clear  that  the  relative  importance  of  random  fluctuations 
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can  be  reduced  by  averaging  the  data,  thereby  separating  the 
•signals”  of  interest  from  the  "noise.”  Autocorrelation  and 
ipectral  analysis  are  complementary  averaging  techniques  which 
are  powerful  tools  in  extracting  the  non-random  components  from 
a  time  series. 

In  Fig.  18  is  a  particular  realization  of  a  stochastic 
process  with  the  number  of  observations,  N  =  80.  Visual  inspec¬ 
tion  provides  few  clues  for  deciding  whether  any  periodicities 
are  present.  However,  there  does  appear  to  be  some  slowly 
varying  component. 

A  first  step  in  determining  the  nature  of  the  non-random 

component  is  to  estimate  an  autocorrelation  function  from  the 

observed  time  series.  Autocorrelation  is  accomplished  by 

correlating  the  original  series  with  another  series  composed 

of  the  original  one,  but  moved  or  slipped  in  sequence  by 
« 

0,1,2, ...  ,K  observations.  The  numbers  0,1,2,...,K  are  called 

the  lag  numbers.  Therefore,  the  value  of  the  autocorrelation 

function  at  lag  0  is  1,0,  which  is  the  product  moment  correlation 

of  the  series  with  itself.  Likewise  the  value  of  the  estimated 

autocorrelation  function  at  lag  1  is  the  correlation  of  the 
series  with  a  new  series  made  up  of  the  original 
original/one,  but  starting  at  the  second  observation.  For  a 

discrete  time  series  (i.e.,  one  that  is  composed  of  observations 

•paced  in  time  by  an  amount  At;  At  in  Fig.  18  is  12.5  msec.) 

*bo  autocorrelation  is  the  normalized  version  of  the  autocovar- 

lflsce  function;  The  autocovariance  function  can  be  estimated 
lyi 


Figure  18.  A  stochastic  process  with  a  slowly  oscillating 


component. 


90 


N-K 

(1)  Cxx(K)  =  |  l  (xt-x>  (xt+k-x)  K=0,1,2,...,N-K 

•  '  t=l 

where  K  is  the  lag  number,  N  is  the  number  of  observations, 
is  an  observation  at  time  t,  and  x  is  given  by: 


Jk  n 

X  =  N  Z 
K  t=l 


which  is  the  sample  mean  of  the  whole  series.  The  autocorrela¬ 
tion  is  given  by: 


(3)  r....(K)  -  Cxx(K) 

c^roj 


One  can  see  that  the  autocorrelation  function  is  actually  the 

autocovariance  function  divided  by  the  sample  variance  of  the 

series.  The  primary  reason  for  using  the  autocorrelation 

» 

function  instead  of  the  autocovariance  function  is  that  it 
allows  one  to  compare  results  obtained  from  observations  based 
on  different  scales  of  measurement. 

The  estimated  autocorrelation  function  for  the  time  series 
Of  Fig,  18  is  plotted  in  Fig.  19.  The  estimates  were  computed 
for  K  o  0,1, ... ,32.  There  appears  to  be  a  slow  oscillation  in 
this  function;  its  period  is  about  15-17  observations,  or  about 
0.188  sec. -0.213  sec.  (5.32-4.71  cps).  Further,  it  should  be 
noted  that  the  absolute  value  of  the  correlations,  on  the 
•for age, decrease  as  the  number  of  lags  increase.  This  latter 
effect  is  the  result  of  the  random  components  of  the  original 
time  series,  whereas  the  obvious  oscillation  in  the  auto¬ 
correlation  function  is  the  result  of  some  underlying  periodic 
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Figure  19.  Autocorrelation  function  based  on  32  lags. 

This  function  was  estimated  from  the  data  presented  in  Fig.  18. 


\ 


process.  Actually  the  data  in  Fig.  18  were  generated  for  the 
purposes  of  demonstration  by  a  digital  computer,  according  to 
the  following  scheme: 

x(t)  *=  a  cos  (2iTfot)  +  z  (t) 

where  a  *  1.0,  f  =  5.0  cps  or  0.0625  cycles/12.5  milliseconds 
z (t)  is  a  sample  of  white  noise  with  zero  mean  and  unit  variance. 
Thus  autocorrelation  was,  at  least  partially,  successful  in 

t 

extracting  the  process  of  interest  from  the  random  fluctuations. 

For  purposes  of  comparison  it  is  important  to  note  that  the 
autocorrelation  function  for  white  noise  would  be  close  to  zero 
at  all  lags.  Also  the  longer  the  record  of  normal  deviates 
being  examined  (i.e.,  the  larger  the  sample),  the  closer  the  auto¬ 
correlation  function  approximates  zero,  except,  of  course,  at 
lag  0  where  it  would  still  be  1. 

The  validity  of  the  autocorrelation  function  in  describing 
a  stochastic  process  depends  on  the  assumption  of  "weak  station¬ 
arity"  or  "covariance  stationarity."  These  equivalent  terms 
represent  a  degree  of  statistical  stability  intermediate 
between  non-stationarity  and  complete  stationarity.  A  process 
Is  said  to  be  completely  stationary  if  and  only  if  the  mean 
variance  and  higher  moments  do  not  change  in  the  long  run .  The 
assumption  of  weak,  or  covariance  stationarity  holds  when  the 
nean  and  variance  do  not  change  in  the  long  run,  but  the  higher 
foments  may  or  may  not  be  changing.  A  process  is  non-s tat ionary 
either  or  both  the  mean  and  variance  do  not  remain  constant. 
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Thus  a  series  of  observations  displaying  an  obvious  linear  trend 
is  non- stationary.  The  assumption  of  weak  stationarity  is 
sufficient  to  imply  that  the  autocovariance  function  depends  only 
on  time  separation;  i.e.,  since  the  variance  is  constant  for  a 
•given  series  of  observations  the  covariance  terms,  of  which  the 
autocovariance  function  is  composed,  depended  only  on  the 
structure  (if  any)  of  the  variations  of  the  individual  data 
points.  The  cosine  plus  noise  process  is  (in  theory,  at  least) 
completely  stationary;  thus,  it  is  also  weakly  stationary. 

If  one  can  assume  that  a  stochastic  process  is  weakly  stationary 
or  covariance  stationary,  then  autocorrelation  and  spectral 
analysis  are  valid  techniques  for  describing  the  process. 

The  stochastic  processes  encountered  in  actual  practice 
are.  not  nearly  so  orderly  as  the  contrived  example.  Therefore, 
it  is  desirable  to  have  some  method  for  erecting  confidence 
intervals  about  the  autocorrelation  function.  Unfortunately, 
adequate  procedures  for  accomplishing  this  have  not  been  worked 
out.  However,  the  statistical  properties  of  the  sample-smoothed 
spectral  density  function  (the  Fourier  transform  of  the  weighted 
autocorrelation  function)  are  rather  well  known  (see  e.g,, 

'Jenkins  &  Watts,  1968) . 

Before  going  on  to  a  brief  explanation  of  spectral  analysis, 
will  be  necessary  to  consider  the  topic  of  Fourier  analysis. 
Courier  techniques  allow  one  to  express  time-domain  functions 
*n  terms  of  their  frequency  (cycles  per  unit  time)  components. 
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Transformation  from  the  time  domain  into  the  frequency  domain 
is  accomplished  principally  by  application  of  Fourier's  theorem, 
which  states  that  a  given  analytic  function  can  be  approximated 
to  any  degree  of  accuracy  by  a  linear  combination  of  sine  and 
cosine  functions.  The  spectrum  of  a  process  or  function 
reveals  the  way  in  which  different  frequencies  of  oscillation 
contribute  to  the  variance.  The  spectrum  is  the  Fourier  transform 
of  some  function  in  the  time  domain.  In  practice  one  deals  with 
functions  which  have  a  finite  extent  in  time,  and  are  sampled 
at  a  spacing  At  in  time.  These  restrictions  have  a  number  of 
effects  on  the  Fourier  transform.  A  finite  length  record 
results  in  distortions  in  the  frequency  domain.  Loosely  speak¬ 
ing,  these  distortions  become  worse  as  the  length  of  the 
observed  series  gets  shorter.  Further,  it  is  generally  true 
that  the  degree  to  which  peaks  in  the  spectrum  can  be  resolved 
depends  on  the  length  of  the  record  -  the  longer  the  record  the 
greater  the  resolution,  i.e.,  the  higher  and  narrower  the  peaks 
become.  Sampling  observations  every  At  units  of  time  auto- 
tmtically  limits  the  upper  range  of  frequencies  which  can  be 
represented  in  the  spectrum  of  a  realizable  prodess.  The 
highest  rate  of  oscillation  that  can  be  detected  in  any  record 
*3  *5  cycles/At  (Nyquist  frequency),  since  frequencies  higher 
than  this  go  through  their  oscillations  between  the  points  of 
observation,  and  thus  can  only  be  "seen"  as  lower  frequencies 
(^UasosM). 
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-Special  difficulties  also  arise  when  Fourier  analysis  is 
applied  to  a  stochastic  process.  The  theoretical  spectrum  for 
a  normal  process  or  white  noise  is  a  flat  line  of  zero  slope.. 

This  seems  intuitively  reasonable  because  a  continuous  range 
of  frequencies  is  present  in  white  noise  and  each  contributes 
equally  to  the  variance.  Unfortunately ,  the  raw  spectral 
estimates  for  a  normal  process  do  not  converge  in  a  statistical 
sense  as  the  length  of  the  record  is  increased,  although  the 
spectrum  does  maintain  the  tendency  to  have  zero  slope.  How¬ 
ever,  a  smoothed  spectral  estimator  does  converge  statistically, 
in  that  the  variance  of  the  spectral  estimates  decrease  as  more 
smoothing  is  carried  out.  But  smoothing,  by  increasing  the 
bias,  may  defeat  the  very  purpose  of  carrying  out  the  analysis  - 
detection  and  resolution  of  peaks.  Therefore,  when  computing 
smoothed  spectra  of  a  stochastic  process,  one  must  compromise 
between  variance  and  bias  (stability  and  fidelity) .  Jenkins  & 
Watts  (1968)  suggest  an  empirical  approach  to  accomplish 
this  compromise.  It  involves  estimating  the  spectral  density 
function  from  at  least. three  autocorrelation  functions 
truncated  at  three  different  lag  values.  Smoothing  in  the 
frequency  domain  is  equivalent  to  multiplying  the  autocorrelation 

at 

function  by  a  lag  window  or  weighting  function.  The  lag  window 
°r  smoothing  procedure  used  in  the  analysis  that  follows  is 
called  the  Tukey  window  (also  Tukey-Hanning  window)  and  is 
9 Won  by: 
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(4)  wT(u)  = 


1/2  (1  +  cos  iru/M) ,  ju|<  M 
°t  | u ( >  M 


where  M  is  the  number  of  lags  used  to  estimate  the  autocorrela¬ 
tion  function.  (The  "window"  nomenclature  was  suggested  by 
Blackman  &  Tukey,  1958. )  It  can  be  shown  that  the  smoothed 
spectral  estimates  are  distributed  approximately  as  x2  with 
the  degrees  of  freedom  v  >  2,  (See  Jenkins  &  Watts,  1968, 
Chapter  6.)  In  general,  the  number  of  degrees  of  freedom 
associated  with  a  spectral  estimate  depends  on  the  length  of 
the  record,  the  type  of  smoothing  procedure,  and  the  number 
of  lags  used  in  estimating  the  autocorrelation  function.  For 


the  Tukey  window  the  degrees  of  freedom  are  gi^en  by: 
(5)  v  «  2.667  £ 

n 


i 


* 


l 
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whore  M  is  the  number  of  lags  and  T  is  the  length  of  the  record. 
Jenkins  &  Watts  (1968)  present  graphs  from  which  the  confidence 
intervals  for  an  estimated  spectral  density  function  can 
60  Convoniontly  obtained  once  the  degrees  of  freedom  are  known. 
Tho  bandwidth,  b,  for  the  Tukey  window  is:  i 

(6)  b  <= 

4 

gives  an  indication  of  the  range  of  frequencies  over  which 

I  L  . 

spectral  estimates  have  been  smoothed.  Generally  speaking, 
lh°  bandwidth  times  the  variance  of  the  estimate  is  a  constant. 
*Ua  a  small  bandwidth  implies  a  large  variance  of  the  estimate 
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but  small  bias;  conversely,  a  wide  bandwidth  means  a  small 
variance  and  large  number  of  degrees  of  freedom  but  a  large 
bias.  For  the  Tukey  window,  estimates  which  are  more  than  one 
bandwidth  apart  are  essentially  statistically  independent. 

In  Fig.  20  are  the  estimated,  smoothed  spectral  density 
functions  for  the  time  series  in  Fig.  18.  The  estimates 
were  obtained  using  the  formula: 

M-l 

(7)  Sxx(f)  =  2A  trxxt°)  +  £  WT(K)  rxx(K)  cos  (irKf/W 

K=1 

where  f  *»  0,1,2, ...  ,100,  &  is  the  spacing  of  observations,  in 

this  case  h  «  12.5  msec.,  M  is  number  of  lags  upon  which  the 

autocorrelation  functions  are  based.  For  very  long  series  it 

is  preferable  to  use  the  fast  Fourier  transform  because  it 

considerably  reduces  computation  time  (Cochran  et  al.,  1967). 

Each  estimated  function  is  the  Fourier  transform  of  the  weighted 

autocorrdation  function  truncated  at  three  different  lag  values. 

(Multiplying  the  autocorrelation  function  by  the  lag  window  is 

equivalent  to  smoothing  in  the  frequency  domain.)  The  reason 

for  estimating  three  functions  might  have  been  anticipated  by 

fho  foregoing,  and  should  be  clarified  by  the  ensuing  discussion. 

% 

f'cr  the  moment  it  is  important  to  notice  several  features  of 
pig.  20.  The  range  of  frequencies  extends  from  0  to  40  cps, 
ftt*d  the  smoothed  spectral  estimates  are  plotted  on  a  logarithm 
•‘fc  scale.  The  largest  peak  in  each  of  the  three  spectral 
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Figure  20.  Spectral  density  functions  of  the  cosine  plus 
noise  process  estimated  from  the  autocorrelation  function  shown 
in  Fig.  19.  The  numbers  labeling  each  curve  are  the  truncation 
points  of  tho  particular  version  of  the  autocorrelation  function 
from  which  the  estimates  were  obtained.  The  ordinate  represents 
log  IQ  relative  contribution  to  the  variance.  The  abscissa 
is  frequency  of  oscillation  in  cycles  per  socond. 
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density  functions  is  in  the  low  frequency  range.  For  the 
functions  estimated  from  16  and  32  lags  the  largest  peaks  are 
centered  at  5.0  cps.  This  is  the  frequency  of  oscillation  that 
was  purposely  mixed  with  the  random  deviates  to  obtain  the 
original  series  shown  in  Fig.  18.  The  various  spurious  peaks 
that  appear  in  the  higher  frequencies  for  the  16  and  32  lag 
functions  demonstrate  the  tendency  of  the  variance  of  the 
estimates  to  increase  as  the  bandwidth  is  decreased. 

The  .95  confidence  intervals  and  the  corresponding  band- 
widths  for  the  three  functions  are  also  shown  in  Fig.  20. 
Reproduced  in  Fig.  21  is  the  estimated  spectral  density  • 
function  for  8  lags  with  the  .95  confidence  interval  erected 
about  it.  The  true  spectral  density  function  should  lie  in 
this  interval  95%  of  the  time,  The  heavy  black  line  of  zero 
slope  represents  the  spectral  density  function  of  white  noise. 
Moreover,  it  is  clear  that  the  true  function  might  be  flat. 

If  it  were  not  known  in  advance  that  there  really  is  an  under¬ 
lying  periodic  process  (since  it  was  purposely  contrived) , 

one  would  be  uncertain  as  to  the  true  nature  of  the  process. 

\ 

This  uncertainty  could  be  reduced  if  the  process  were  ob¬ 
served  for  a  longer  period  of  time,  thereby  increasing  the 
number  of  observations  in  the  record.  Increasing  the  length  of 
the  record  has  two  effects.  First,  it  increases  the  degrees 
°f  freedom, thus  tightening  the  confidence  interval  for  a 
9ivon  bandwidth.  Second,  more  observations  tend  to  yield  a 


Figure  21.  Spectral  density  function  with  95%  confidence 
erected  about  it.  The  curve  labeled  "8"  is  reproduced  from 
Fig.  20.  The  dashed  curves  above  and  below  this  function 
represent  the  95%  confidence  interval.  The  flat  heavy  black 
line  is  the  theoretical  spectral  density  function  for  white 
noise  or  a  Gaussian  process. 
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more  stable  autocorrelation  function  which  in  turn  causes 
fewer  distortions  when  transformed  into  the  frequency  domain. 

In  Fig.  22  are  the  spectral  density  functions  for  the  same 
cosine  plus  noise  process,  but  with  N  =  160  observations.  The 
confidence  intervals  are  much  narrower  than  in  Fig.  20. 

Further,  spurious  peaks  still  occur,  but  to  a  lesser  degree 
than  before.  Moreover,  these  false  peaks  are  not  as  high  as 
in  Fig.  20,  nor  are  they  centered  at  the  same  frequencies. 

At  this  juncture,  it  is  appropriate  to  discuss  the  matter 
of  optimal  record  length.  The  advantages  of  examining  a  series 
made  up  of  many  observations  which  yields  stable  estimates 
must  be  weighed  against  the  ensuing  disadvantages  arising  from 
the  greater  time  and  expense  (both  experimental  and  computational) 
that  long  records  entail.  Additionally,  the  process  of  interest 
nay  remain  covariance  stationary  only  for  a  short  time.  Thus 
the  length  of  the  time  series  chosen  cannot  be  specified  in 
general.  It  should  include  enough  observations  to  provide 
reasonable  stability  and  fidelity.  Of  course,  the  highest 
frequency  of  interest  determines  the  choice  of  At.  The  pro¬ 
cedure  of  estimating  three  spectral  density  functions  from 

4 

each  sample  time  series  is  an  empirical  method  suggested  by 
-Jenkins  4  Watts  (1968)  for  gaining  some  insight  into  the 
stability  of  the  estimates.  In  many  practical  situations 
te-g.,  the  analysis  which  follows)  rather  short  scries  (N  =  60 
0r  V0)  can  bo  analyzed  to  determine  the  general  shape  of  the 


Figure  22.  Spectral  density  functions  of  the  cosine  plus 
noise  process  estimated  from  autocorrelation  functions  which 
were  themselves  computed  from  a  series  of  160  observations. 
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spectral  density  function;  however,  rarely  can  individual  peaks 
be  resolved  from  so  few  observations,  unless  the  process 
produces  very  stable  estimates. 

Application  of  Spectral  Analysis  to  Session  Records  of  Peak  Force 
In  Chapters  3  and  4  a  training  session  was  terminated  when 
a  subject  had  obtained  50  reinforcements.  This  procedure 
allowed  individual  differences  to  arise  in  terms  of  the  number 
of  responses  recorded  for  each  subject  during  a  session.  Thus 
during  Phase  II  the  number  of  responses  ranged  from  52  to  over 
100.  These  sequences  of  peak  force  measurements  were  the  data 
to  which  spectral  analysis  was  applied.  Since  the  behavior 
was  free-operant,  time  between  measurement  was  left  free  to 
vary  according  to  the  characteristics  of  each  subject.  As  a 
result,  the  data  cannot  be  expressed  in  terms  of  real  time 
units,  rather  peak  force  is  taken  to  be  a  function  of  ordinal 
position  or  number  of  the  response  measurement  in  the  sequence 
of  observations  which  comprise  a  session  record.  Accordingly, 
the  frequencies  of  oscillation  are  expressed  in  units  of  cycles 
per  response  (cpr) , 

\ 

.  It  was  anticipated  that  substantial  individual  differences 
in. patterns  of  responding  would  arise.  Therefore,  no  attempt 
Was  made  to  average  the  computed  spectral  density  functions, 
because  the  number  of  OQservations  was  extremely  small  for  a 
8poctral  analysis,  several  density  functions  were  estimated  for 
cach  subject  using  a  relatively  small  number  of  lags  (4  to 
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16).  Confidence  intervals  were  relied  upon  for  deciding  the 
shape  of  the  spectrum.  Further,  the  notion  of  replication  was 
used  as  an  adjunct  to  the  confidence  intervals  in  the  following 
way:  Spectral  density  functions  for  all  subjects  were  estimated 
for  the  first  two  and  last  two  sessions  of  Phase  II.  Then  the 
functions  for  individual  subjects  on  successive  sessions  (e.g. , 
sessions  1  and  2)  were  compared  visually,  and  similarities  in 
the  slope  of  the  spectrum  were  noted. 

Computations  were  made  by  a  digital  computer  using  the 
formulae  presented  previously,  except  f  in  formula  <7)  varied 
from  0  to  50.  Before  the  autocorrelation  and  spectral  density 
functions  were  estimated,  the  data  were  subjected  to  log1Q 
transformation  and  linear  trend" removal.  Further,  inter¬ 
pretation  of  the  resulting  functions  was  attempted  only  if  the 
assumption  of  weak  stationarity  (or  covariance  stationarity 
was  thought  to  hold . ) 

The  covariance  stationarity  assumption  was  tested  in  the 
following  way:  The  variance  of  the  first  half  of  the  trans¬ 
formed  series  was  compared  with  the  second  half* variance.  F 
ratios  were  then  computed.  Near  equality  of  the  variances  was 
taken  as  evidence  of  covariance  stationarity.  Hov/evcr,  signi¬ 
ficant  departures  suggested  possible  failure  of  the  assumption. 
But  in  view  of  the  large  number  of  variance  ratios  computed, 
ft  was  very  likely  that  some  significant  values  would  obtained. 
Therefore,  the  direction  of  change  in  variance  on  successive 
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sessions  was  also  noted. 

Of  the  96  F  ratios  computed  for  the  first  two  sessions  of 
phase  II ,  12  were  significant  at  the  .05  level — a  larger  number 
than  would  be  expected  on  the  basis  of  sampling  fluctuations 
(see  Table  9).  Likewise,  for  sessions  14  and  15  there  were  24 
such  significant  F  ratios  out  of  96  computed.  (Data  for  one 
subject  were  lost  in  a  card  reader  malfunction) .  Thus  there  was 
some  evidence  that  many  of  the  records  were  not  stationary. 
However,  it  will  be  recalled  that  the  log^g  transformation  was 
only  partially  successful  in  normalizing  the  data.  Since  the 
F-test  is  very  sensitive  to  failure  of  the  normality  assumption 
(Hays,  1963),  it  is  possible  that  the  larger  number  of  signi¬ 
ficant  ratios  resulted  from  non-normal  nature  of  the  transformed 
distributions.  Nevertheless,  the  decision  was  made  to  adopt 
ft  very  conservative  criterion  for  covariance  stationarity .  Thus 
ftny  subject's  data  displaying  a  significant  F  ratio  on  one  of 
two  successive  sessions  was  treated  as  non-stationary.  One 
consideration  which  weighed  heavily  in  choosing  a  strict 
criterion  was  the  fact  that  very  short  series  were  being  analyz¬ 
ed— a  circumstance  which  tends  to  limit  the  validity  of  spectral 
•nalytic  techniques. 

With  respect  to  direction  of  variance  change  during  a 

•csoion,  about  half  of  the  subjects  increased  and  half  decreased. 

« 

Table  9  are  given  the  number  of  occurrences  of  each  type  of 
tariance  change.  Both  directions  and  degree  of  variance  change 


Table  9 


Number  of  Subjects  Displaying  Each  Type 
of  Variance  Change  During  Phase  II 

i 


Session  Group 

Increase* 

Decrease 

No  Change 

Signif . 

dumber 

• 

at  .  05 
Level 

1 

2 

5 

.0 

1 

2 

6 

2 

o  ; 

1 

1  3 

4 

3 

0 

0 

4 

4 

3 

0 

0 

5 

6 

3 

0 

0 

6 

6 

4 

0 

3 

Totals 

28 

20 

0 

5 

1 

2 

5 

O' 

1 

2 

4 

4 

0 

2 

2  3 

4 

3 

l 

0 

4 

•  3 

3 

0 

1 

.  5 

7 

2 

0 

0 

6* 

5 

5 

•  •  0 

3 

Totals 

25 

22 

l 

7 

1 

2 

4 

i 

0 

2 

5 

2 

0 

3 

14  3 

5 

3 

0 

2 

4 

4 

3 

0 

3 

5 

4 

5 

0 

0 

6 

6 

4 

0 

4 

Totals 

26 

21 

1 

12 

1 

2 

5 

0 

3 

2 

4 

3 

t° 

1 

15  3 

6 

2 

0 

0 

4 

2 

5 

0 

1 

5 

3 

6 

0 

•  4 

6 

6 

4 

0 

3 

Totals 

23 

25 

0 

12 

*otess 

1.  Increase  means  that  variance  of  second  half  was  greater 
than  the  variance  in  the  first  half  of  the  session. 
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yere  approximately  equally  distributed  among  experimental 
groups . 

Early  Conditioning  in  Phase  II 

For  sessions  1  and  2  of  Phase  II,  37  subjects  were  assumed 
to  have  covariance  stationary  records.  Only  eight  of  these  dis¬ 
played  spectral  density  functions  which  were  substantially 
different  from  a  Gaussian  process  on  successive  sessions. 
Interestingly,  the  type  of  non-random  component  was  similar  for 
all  eight  subjects.  The  spectral  density  functions  for  these 
animals  provided  evidence  for  an  underlying  high  frequency 
component  of  1/3  to  1/2  cycle  per  response.  In  Figs.  23  and  24 
are  presented  the  data  for  one  of  these  subjects.  Figure  23-A 
shows  the  peak  force  of  response  plotted  as  a  function  of 
ordinal  position.  There  does  seem  to  be  a  distinct  tendency 
for  the  forces  to  "oscillate1'  rapidly  up  and  down.  This  same 
pattern  is  again  seen  in  the  autocorrelation  function  (Fig. 

23-b) .  And,  as  expected,  the  estimated  spectral  density  function 
(Fig.  23-C)  shows  that  higher  frequencies  account  for  more  of  the 
variance  than  do  the  lower  frequencies.  The  high  frequencies 
also  dominate  the  pattern  of  peak  forces  emitted  during  session 
2  (see  Fig.  24-A,  B,  C).  Estimates  of  the  spectral  density 
function,  for  this  session  are  quite  stable  (Fig.  24-C) :  the 
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Figure  23-A.  Peak  force  of  response  as  a  function  of 
ordinal  position.  The  data  are  from  the  first  session  of  Phase 
XI.  This  individual  was  in  Group  4  (see  Chapter  3) . 


\ 


(tuB)  3DH0J  MV3d 


Figure  23-B.  Autocorrelation  function  of  16  lags  estimated 


from  the  data  shown  in  Fig.  23-A.  Note  the  relatively  large 
negative  correlation  at  lag  1. 
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Figure  23-C.  Spectral  density  functions  for  the  data 
presented  in  Fig.  23-A.  The  ordinate  is  log^g  relative  pro¬ 
portion  of  variance  accounted  for  by  the  frequencies  of  oscilla 
tion  which  are  marked  off  on  the  abscissa.  The  frequency  units 
are  cycles  per  response.  Thus  the  range  of  frequencies  extends 
from  0  to  .5  cycles  per  response.  For  example,  an  oscillation 
which  requires  two  responses  for  completion  has  a  frequency  of 
.5  cycles  per  response.  ~ 
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Figure  24-A.  Peak  force  of  response  as  a  function  of 
ordinal  position  for  the  same  subject  shown  in  Figs.  23~A,  B, 
and  C.  These  data  were  recorded  in  the  second  session  of  Phase 
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Figure  24-C.  Spectral  density  functions  computed  from  the 
session  peak  force  data  shown  in  Fig.  24-A.  Note  the  positive 
slope  of  all  three  functions.  Increasing  the  number  of  lags 
from  4  to  16  only  slightly  influences  the  shape  of  the 
function — a  fact  which  indicates  a  high  degree  of  stability. 
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shape  of  the  function  changes  very  little  as  the  lag  number  is 
increased  from  4  to  16.  The  other  seven  subjects  displayed  a 
replicable  pattern  similar  in  shape  and  magnitude  to  the  indi¬ 
vidual  just  discussed.  In  terms  of  the  reinforcement  contin¬ 
gencies,  of  the  eight  subjects  exhibiting  a  repeatable  pattern 
all  but  one  belonged  to  the  groups  which  received  the  highest 
reinforcement  densities  (mg/min)  i.e.,  Groups  3,  4,  5.  However, 

this  tendency  to  oscillate  rapidly  in  F  is  not  confined  to  the 

P 

groups  exposed  to  the  force-correlated  sweetness  schedules,  since 
three  of  the  "high  frequency"  subjects  were  in  Group  4.  Be¬ 
cause  there  were  so  few  subjects  displaying  this  reliable 
tendency  to  oscillate  rapidly,  any  firm  conclusion  must  await 
further  experimental  analysis. 

The  shapes  of  the  spectral  density  functions  shown  in 
Fig.  23-C  and  Fig.  24-C  are  very  similar  to  a  first  order  auto¬ 
regressive  process  or  Markov  process.-  This  kind  of  theoretical 
process  has  the  characteristic  that  the  value  of  observation 
n+1  depends  only  on  the  value  of  observation  n,  and  is  inde¬ 
pendent  of  all  other  previous  values.  It  is  tempting  to  speculate 
that  what  occurs  on  one  response  influences  only  the  next  response, 
indicating,  on  a  most  conjectural  level,  a  brief  "memory"  span. 
Without  more  quantitative  support  such  suggestions,  hbwever, 
must  remain  purely  speculative. 

In  contrast  to  the  eight  atypical  subjects,  (i.e.,  those 
displaying  a  replicable  high  frequency  oscillation)  were  the 
remaining  29  animals,  which  also  exhibited  covariance  stationary 
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Fp  records.  Spectral  density  functions  for  these  animals  were 


essentially  flat,  with  no  particular  range  of  frequencies  tend¬ 


ing  to  dominate.  Further,  neither  the  slopes  nor  the  peaks  of 


these  functions  were  the  same  on  successive  sessions.  The 


data  for  a  typical  individual  displaying  these  qualities  are 


presented  in  Fig.  25  (A,  B,  and  C)  and  Fig.  26  (A,  B,  and  C) . 


These  results  are  compatible  with  the  conclusion  that  the 


"pattern"  of  peak  force  for  most  of  the  animals  does  not  depart 


substantially  from  complete  randomness.  This  is  rather  sur¬ 


prising  for  early  conditioning.  One  might  have  expected  some 


kind  of  intrasession  effect  to  emerge  in  the  early  sessions. 


but — on  the  whole — the  data  do  not  support  this  view. 


Later  Conditioning  in  Phase  II 


Only  27  subjects  met  the  criterion  for  covariance' 


stationarity  established  previously.  Of  these,  six  yielded 


evidence  of  a  repeatable,  non-random  pattern  of  responding.  The 


particular  shapes  of  the  spectral  density  functions  for  these 


atypical  animals  were  quite  similar  to  those  described  for  the 


data  collected  early  in  conditioning.  Further,  these  six 
subjects  were  approximately  equally  represented  in  each  of  the 


experimental  groups.  However,  these  six  subjects  were  not 
the  same  individuals  exhibiting  non-randomness  in  the  first 


sessions. 


The  failure  for  individual  differences  to  remain  apparent 
throughout  conditioning  may  become  more  understandable  by  some 
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Figure  25-A.  Peak  force  as  a  function  of  ordinal  position 
within  a  session.  These  data  were  gathered  from  an  individual 
in  Group  2  during  the  first  session  of  Phase  II. 


t 


Figure  25-B.  Autocorrelation  function  of  16  lags  esti¬ 
mated  from  the  data  presented  in  Fig.  25-A. 


t 


Figure  25-C.  Spectral  density  functions  obtained  from 
the  session  peak  force  record  shown  in  Fig.  25-A.  The  functions 
are  very  flat  suggesting  that  all  frequencies  are  approximately 
equally  represented  in  the  process.  Compare  the  functions  for 
4  and  8  lags  with  the  theoretical  function  for  white  noise 
shown  in  Fig.  21. 


Figure  26-A.  Peak  force  as  a  function  of  ordinal  position 
for  the  second  session  of  Phase  II.  This  is  the  same  individual 
whose  data  are  given  in  Fig.  25-A. 
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Figure  26-C.  Spectral  density  functions  for  a  subject 

of  Group  2  estimated  from  data  obtained  in  the  second  session 
of  Phase  II. 


QUENCY  (cycles/response) 
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additional  considerations.  For  both  early  and  late  conditioning, 
aiany  subjects  which  did  not  display  repeatable  patterns,  never¬ 
theless  exhibited  a  strong  high  frequency  component  on  one  of 
the  two  sessions.  Moreover,  a  large  number  of  animals  which 
produced  non-stationary  records  in  one  session  shifted  to 
covariance  stationary  responding  in  the  subsequent  session,  or 
vice  versa.  If  one  considers  all  the  peak  force  records  that 
yielded  non-significant  F  ratios,  the  dominant  non-random 
tendency  emerging  in  the  spectral  density  functions  was  in  the 
high  frequency  region.  It  is  still  true,  however,  that  the 
pattern  of  forces  emitted  by  the  large  majority  of  subjects, 
during  any  given  session,  did  not  depart  substantially  from 
randomness.  From  the  foregoing  statements  one  can  infer  that 
there  may  be  a  genuine  tendency  for  most  of  the  animals  to 
chow  rapid  ups  and  downs  in  the  emission  of  forces,  but  only 
rarely  do  the  high  frequencies  dominate  to  the  extent  that  they 
emerge  as  a  non-random  component.  On  the  other  hand,  one  cannot 
erguo  in  the  same  manner  for  the  intermediate  and  lower  fre¬ 
quencies  because  these  components  mever  appear  with  any 
strength  or  regularity. 

Overview  of  Possible  Feedback  Processes 

The  sequences  of  forces  emitted  within  a  given  session  did 
n°t  display  any  linear  trends.  When  examined  in  this  way,  the 
Ptak  force  data  do  not  provide  for  a  description  of  the  supposed 
^j}tiation-hunqcr  loop.  Either  peak  force  is  unresponsive  to 
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such  satiation  effects,  or  a  much  longer  time  of  responding  is 

t 

required  before  these  effects  can  be  observed. 

Spectral  analytic  techniques  suggested  that  there  may  be 
a  high  frequency  (.33  to  .5  cpr)  oscillation,  but  it  is  rel¬ 
atively  insignficant  when  compared  to  the  pervasiveness  of  the 
random  nature  of  the  resporse  sequences.  Nevertheless,  this 
high  frequency  component  does  provide  a  modicum  of  evidence 
for  a  reinforcement-response  loop.  The  tendency  for  forces  to 
oscillate  rapidly  up  and  down  may  be  interpreted  in  the  follow¬ 
ing  way:  The  rat  is  exposed  to  successive  changes  in  sweetness 
or  amount  (or  both)  that  are  probably  discriminably  different. 
Thus  receipt  of  an  8.0%  solution  would  increase  the  likelihood 
that  the  next  response  would  be  harder,  and  if  a  24.0%  solu¬ 
tion  was  then  obtained,  one  would  expect  the  ensuing  response 
to  be  less  vigorous.  This  argument  follows  from  the  empirical 

relations  between  mean  F  and  amount  of  reinforcement  mentioned 

P 

earlier  (see  DiLollo,  Ensminger,  &  Notterman,  1965?  and 
Notterman  &  Mintz,  1965).  The  same  reasoning  holds  for  subjects 

f  J 

in  Group  1,  except  only  two  different  reinforcements  are  given — 
0%  and  8.0%.  * 

From  a  quantitative  point-of-view,  the  results  of  the 
foregoing  analyses  have  some  important  implications.  First,  it 
may  be  appropriate  to  average  the  spectral  estimates  to  obtain 
a  clearer  picture  of  the  high  frequency  component.  Second,  in 
future  related  experiments  the  number  of  reinforcements 
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delivered  in  one  session  can  probably  be  increased  to  75  or 

i 

more  without  endangering  the  stationarity  assumptions.  The 
longer  series  thereby  produced  should  provide  more  stable 
estimates  of  the;  spectral  density  functions.  Third,  the  data 
analyzed  so  far  show  that  very  little  information  is  lost  by 
using  mean  to  describe  session  performance. 


Chapter  6 

. Summary  of  Results  and  Gbnclusions 

A  number  of  conclusions  may  be  based  on  the  foregoing 
experimental  and  quantitative  analyses: 

1.  On  the  average,  proportional  reinforcement  defined 
exclusively  in  terms  of  amount  (mg.)  does  not  shape  forces 
upward.  This  is  true  of  continuous  proportionality  as  well  as 
discontinuous  step-wise  proportionality  under  a  number  of 
parametric  conditions. 

2.  On  the  other  hand,  force  proportional  reinforcement, 
defined  in  terms  of  concentration  of  sucrose  does  substantially 
increase  the  likelihood  of  high  force  emission.  Rats  press 
harder  to  get  sweeter  rewards  but  do  not  press  harder  to  obtain 
more  of  the  same  kind  of  reward. 

3.  Peak  force  distributions  are  virtually  continuous  and 
are  unimodal,  even  for  the  2-step  proportional  concentration 
groups.  There  is  a  general  tendency,  manifested  by  all  the  groups 
(1-6)  for  the  mean  and  standard  deviation  to  be  highly  correlated. 
(Though  not  previously  mentioned,  this  relationship  holds  for 

the  combined  groups  of  Chapter  2  -  Groups  C8,  P8,  and  P32  - 
where  r  =  .691. ) 

4.  Rate  of  responding  viewed  as  interresponse  time 
independently  of  consumption  time,  was  not  differentially 
influenced  by  the  reinforcement  operations,  probably  because  of 
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a  "floor"  effect  -  all  groups  were  responding  about  as  fast  as 
was  consistent  with  consuming  all  the  sucrose  delivered  in  a’ 
single  reinforcement. 

5.  The  ratio  of  payoff  to  exertion,  the  gain,  had  very 
little  influence  on  mean  peak  force.  However,  the  studies  did 
provide  some  evidence  that  higher  gain  values  were  "chosen"  by 
most  subjects,  but  only  when  high  gain  was  paired  with  a  low 
force  requirement. 

6.  Within  a  given  session  there  was  no  evidence  for  a 

linear  trend  in  F  ,  perhaps  indicating  the  absence  of  any 

P 

satiation  effects. 

7.  Also,  the  sequences  of  responses  emitted  during  a 
session  showed  only  slight  departure  from  randomness,  in  the 
sense  that  the  spectral  density  functions  were  generally  flat. 

Such  non-random  effects  that  did  emerge  took  the  form  of  a  high 
frequency  component  (.33  to  .5  cpr) ,  indicating  a  tendency  for 
Fp  to  display  sharp  ups  and  downs.  The  serial  effects  were 
distributed  approximately  equally  among  the  various  experimental 
groups.  With  regard  to  the  question  of  constant  variance  within 
a  given  session,  the  evidence  is  compatible  with  the  notion  that 
the  majority  of  subjects  do  exhibit  stationary  sequences  of  F^. 

It  probably  comes  as  no  surprise  that  the  proportional 
contingency  between  force  of  response  and  concentration  of  sucrose 
brought  about  more  forceful  responding.  This  finding  does, 
however,  servo  to  underscore  the  importance  of  the  stimulus 
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properties  of  reinforcement.  By  way  of  speculation  it  may  be 
suggested  that  the  taste  of  the  various  solutions  were  differ¬ 
ential  cues  which  were  essential  in  the  process  of  learning  to 
press  harder. 


« 
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